Needful Things
#Toolbox
delicate pewter
PyCG generates call graphs for Python code using static analysis. It efficiently supports
Higher order functions
Twisted class inheritance schemes
Automatic discovery of imported modules for further analysis
Nested definitions
https://github.com/vitsalis/pycg
The Open Graph Viz Platform
Gephi is the leading visualization and exploration software for all kinds of graphs and networks. Gephi is open-source and free.
https://gephi.org/
UMAP
Uniform Manifold Approximation and Projection (UMAP) is a dimension reduction technique that can be used for visualisation similarly to t-SNE, but also for general non-linear dimension reduction. The algorithm is founded on three assumptions about the data:
https://github.com/lmcinnes/umap
HDBSCAN
HDBSCAN - Hierarchical Density-Based Spatial Clustering of Applications with Noise. Performs DBSCAN over varying epsilon values and integrates the result to find a clustering that gives the best stability over epsilon. This allows HDBSCAN to find clusters of varying densities (unlike DBSCAN), and be more robust to parameter selection.
In practice this means that HDBSCAN returns a good clustering straight away with little or no parameter tuning -- and the primary parameter, minimum cluster size, is intuitive and easy to select.
HDBSCAN is ideal for exploratory data analysis; it's a fast and robust algorithm that you can trust to return meaningful clusters (if there are any).
https://github.com/scikit-learn-contrib/hdbscan
Top2Vec is an algorithm for topic modeling and semantic search. It automatically detects topics present in text and generates jointly embedded topic, document and word vectors. Once you train the Top2Vec model you can:
Get number of detected topics.
Get topics.
Get topic sizes.
Get hierarchichal topics.
Search topics by keywords.
Search documents by topic.
Search documents by keywords.
Find similar words.
Find similar documents.
Expose model with RESTful-Top2Vec
https://github.com/ddangelov/Top2Vec
The Open Graph Viz Platform
Gephi is the leading visualization and exploration software for all kinds of graphs and networks. Gephi is open-source and free.
Runs on Windows, Mac OS X and Linux.
https://gephi.org/
Neo4j Graph DB
Native graph storage, data science, ML, analytics, and visualization with enterprise-grade security controls to scale your transactional and analytical workloads – without constraints.
There's a free cloud instance
https://neo4j.com/pricing/
Docker
docker run --publish=7474:7474 --publish=7687:7687 --ip=0.0.0.0 neo4j:5.12.0
Diagrams charts etc. Has free desktop
https://app.diagrams.net/
delicate pewter
