Learn how to create synthetic data in case your project runs low on data or use it for simulations — Data is the fuel in Data Science projects. But what if the observations are scarce, expensive, or difficult to measure? Synthetic data can be the solution. Synthetic data is artificially generated data that mimics the statistical properties of real-world events. I will demonstrate how to create continuous synthetic data by…

Create interactive, stand-alone, and visually attractive charts that are built on the graphics of d3 javascript (d3js) but configurable with Python. — Python has become one of the most popular programming languages to analyze and visualize your data. Visualizing can be the key to success in projects because it can reveal hidden insights in the data, and improve understanding. The best way to understand and explain the data is by making it…

Reveal the hidden causal variables in your data set by using the best-suited Bayesian causal inference library: a comparison with hands-on examples of five popular libraries. — Understanding the causal effect of variables in systems or processes is very valuable. There are a number of Python libraries that can assist in determining causal relationships. I will compare five popular causal inference libraries in their functionality, ease of use, and flexibility. Each is accompanied by hands-on examples. The…

Learn how to quantitatively detect which features drive the formation of the clusters — Cluster analysis is a great technique for identifying groups with similar patterns. However, once clusters are formed, it can remain challenging to determine the driving features behind the clusters. But this step is crucial to reveal valuable insights that may have been missed before and can be used for decision-making…

Hidden gems can be found using clustering approaches but you need the right clustering method and evaluation approach to make sensible clusters. Learn how to find them in four steps. — With unsupervised cluster analysis, we can group observations with similar patterns, and reveal (hidden) trends in the data. The use of cluster evaluation methods helps to determine the clustering tendency, quality, and optimal number of clusters. In this blog, we will dive into cluster evaluation methods, learn how to interpret…

Thanks to PCA’s sensitivity, it can be used to detect outliers in multivariate datasets. — Principal Component Analysis (PCA) is a widely used technique for dimensionality reduction while preserving relevant information. Due to its sensitivity, it can also be used to detect outliers in multivariate datasets. Outlier detection can provide early warning signals for abnormal conditions, allowing experts to identify and address issues before they…

Learn how to detect outliers using Probability Density Functions for fast and lightweight models and explainable results. — Anomaly or novelty detection is applicable in a wide range of situations where a clear, early warning of an abnormal condition is required, such as for sensor data, security operations, and fraud detection among others. Due to the nature of the problem, outliers do not present themselves frequently, and due…

A step-by-step guide to effectively release your Python package in the Python Package Index (PyPI) to pip install it — Releasing a package in PyPi and installing it with pip has great advantages as it will automatically resolve dependencies, and it becomes accessible on any machine. However, publishing an open-source Python Package requires some preparation steps, such as the unambiguous structuring of your directory, versioning, archive building, and publishing the…

Scatterplots are extremely useful for visualizing relationships between two sets of numerical variables. It is even more insightful when it is interactive with zooming and brushing capabilities. — The scatter plot is perhaps the most well-known chart to visualize numerical variables. Such basic charts are very useful from time to time, especially when they are interactive with brushing and zooming capabilities. In this blog post, I will demonstrate how to create interactive scatter plots with varying colors, sizes…