Kuldeep Jiwani
Director / Research Data Scientist
Thales (Guavus)
location_on India
Member since 3 years
Kuldeep Jiwani
Specialises In
Researcher, Data Scientist, Data Science Architect, Performance specialist, Entrepreneur
Currently working as a Data Science Researcher and Director in Thales (Guavus) handling various Data Science projects. Along with actively exploring new techniques of Machine Learning via various research projects. Thus ensuring that Data Science could be effectively applied to real world scenarios and able to solve important problems.
The research areas are focused primarily on Unsupervised Machine Learning techniques, where we have to discover information from given data, with no available labels. The world of Supervised ML have made great advances with the advent of Deep Learning. But in majority of industries and real world scenarios, labels are hard to obtain so Unsupervised ML is the only option to proceed.
Data Science Research:
 Researching on new Machine Learning approaches in the domain of Cyber Security, Fraud Detection, Network Operations
 Data Geometry: Constructing topological spaces via semantic similarity between data attributes
 Session discovery by modelling stochastic periods in time series data
 BOD (Behavioural Outlier Detection): Behavioural models to detect Frauds via Unsupervised modelling techniques
 NSBA (Network Service Behaviour Analytics): An unsupervised ML technique to model network service behaviours. This creates statistical Behavioural models to catch abnormal flows in the network.
Data Science Architectures:
 Distributed AI/ML: ML over the edge and over distributed processing
 Building large scale Machine Learning solutions over TBs of data
 Constructing topological spaces over Big Data
 Designing and architecting billion nodes graphs and Ontologies
Was key architect in building a JVM performance monitoring tool AD4J in Auptyma that was acquired by Oracle in 2007. Presented this tool in GIDS (Great India Developer Summit) 2008. Performance specialist in Big Data technologies  Apache Hadoop, Apache Spark, Yarn, Redis, HBase Applying Machine Learning and Machine Intelligence to Big Data problems to do analytics at large scale

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
NonParametric PDF estimation for advanced Anomaly Detection
20 Mins
Talk
Intermediate
Anomaly Detection have been one of most sought after analytical solutions for businesses operating in the domain of Network Operation, Service Operation, Manufacturing etc. and many other sectors where continuity of operations is essential. Any degradation in operational service or an outage, implies high losses and possible customer churn. The data in such real world applications is generally noisy, have complex patterns and often correlated.
There are techniques like AutoEncoders available for modelling complex patterns, but they can't explain the cause in original feature space. The traditional univariate anomaly detection techniques uses the zscore and pvalue methods. These rely upon unimodality and choice of correct parametric form. If assumptions are not satisfied then there would be a high number of FalsePositives and FalseNegatives.
This is where the need for estimating a PDF (Probability Density Function) arises that too without assuming a prior parametric form i.e. NonParametric approach. The PDF needs to be modelled as close to the true distribution as possible. That is it should have a low bias and low variance to avoid oversmoothing and undersmoothing. Only then we would have better chances of identifying true anomalies.
Approaches like KDE  Kernel Density Estimation assist in such nonparametric estimations. As per research the type of kernel has a lesser role to play than the bandwidth for a good PDF estimation. The default bandwidth selection technique used in both Python and R packages oversmooths the PDF and is not suitable for Anomaly Detection.
We will explain another method, where we run optimisation over a cost function based on modelling Gaussian kernel via FFT (Fast Fourier Transform), to obtain the appropriate bandwidth. Then we will show how we can apply it for Anomaly Detection even when the data is multimodal (have multiple peaks) and the distribution can be of any shape.
Based on research paper under publication "Optimal Kernel Density Estimation using FFT based cost function", currently scheduled for ICDM 2020, New York

keyboard_arrow_down
Sessionisation via stochastic periods for root event identification
45 Mins
Talk
Intermediate
In todays world majority of information is generated by self sustaining systems like various kinds of bots, crawlers, servers, various online services, etc. This information is flowing on the axis of time and is generated by these actors under some complex logic. For example, a stream of buy/sell order requests by an Order Gateway in financial world, or a stream of web requests by a monitoring / crawling service in the web world, or may be a hacker's bot sitting on internet and attacking various computers. Although we may not be able to know the motive or intention behind these data sources. But via some unsupervised techniques we can try to infer the pattern or correlate the events based on their multiple occurrences on the axis of time. Associating a chain of events in order of time helps in doing a root event analysis. In certain cases a time ordered correlation and root event identification is good enough to automatically identify signatures of various malicious actors and take appropriate corrective actions to stop cyber attacks, stop malicious social campaigns, etc.
Sessionisation is one such unsupervised technique that tries to find the signal in a stream of events associated with a timestamp. In the ideal world it would resolve to finding periods with a mixture of sinusoidal waves. But for the real world this is a much complex activity, as even the systematic events generated by machines over the internet behave in a much erratic manner. So the notion of a period for a signal also changes in the real world. We can no longer associate it with a number, it has to be treated as a random variable, with expected values and associated variance. Hence we need to model "Stochastic periods" and learn their probability distributions in an unsupervised manner.
The main focus of this talk will be to showcase applied data science techniques to discover stochastic periods. There are many ways to obtain periods in data, so the journey would begin by a walk through of existing techniques like FFT (Fast Fourier Transform) then discuss about Gaussian Mixture Models. After highlighting the short comings of these techniques we will succinctly explain one of the most general nonparametric Bayesian approaches to solve this problem. Without going too deep in the complex math, we will get back to applied data science and discuss a much simpler technique that can solve the same problem if certain assumptions are satisfied.
In this talk we will demonstrate some time based pattern we discovered while working on a security analytics use case that uses Sessionisation. In the talk we will demonstrate such patterns based on an open source malware attack datasets that is available publicly.
Key concepts explained in talk: Sessionisation, Bayesian techniques of Machine Learning, Gaussian Mixture Models, Kernel density estimation, FFT, stochastic periods, probabilistic modelling, Bayesian nonparametric methods

keyboard_arrow_down
Topological space creation and Clustering at BigData scale
45 Mins
Talk
Intermediate
Every data has an inherent natural geometry associated with it. We are generally influenced by how the world visually appears to us and apply the same flat Euclidean geometry to data. The data geometry could be curved, may have holes, distances cannot be defined in all cases. But if we still impose Euclidean geometry on it, then we may be distorting the data space and also destroying the information content inside it.
In the space of BigData world we have to regularly handle TBs of data and extract meaningful information from it. We have to apply many Unsupervised Machine Learning techniques to extract such information from the data. Two important steps in this process is building a topological space that captures the natural geometry of the data and then clustering in that topological space to obtain meaningful clusters.
This talk will walk through "Data Geometry" discovery techniques, first analytically and then via applied Machine learning methods. So that the listeners can take back, hands on techniques of discovering the real geometry of the data. The attendees will be presented with various BigData techniques along with showcasing Apache Spark code on how to build data geometry over massive data lakes.

No more submissions exist.

No more submissions exist.