Data mining is the computational process of discovering patterns in large data sets involving methods at the intersection of artificial intelligence,machine learning, statistics, and database systems.
It is sometimes called Knowledge Discovery in Databases (KDD) .
The Knowledge Discovery in Databases (KDD) process is commonly defined with the stages:
(1) Selection
(2) Pre-processing
(3) Transformation
(4) Data Mining
(5) Interpretation/Evaluation
Data mining involves six common classes of tasks:
1)Anomaly detection (Outlier/change/deviation detection) – The identification of unusual data records, that might be interesting or data errors that require further investigation.
2)Association rule learning (Dependency modeling) – Searches for relationships between variables. For example a supermarket might gather data on customer purchasing habits. Using association rule learning, the supermarket can determine which products are frequently bought together and use this information for marketing purposes. This is sometimes referred to as market basket analysis.
3)Clustering – is the task of discovering groups and structures in the data that are in some way or another "similar", without using known structures in the data.
4)Classification – is the task of generalizing known structure to apply to new data. For example, an e-mail program might attempt to classify an e-mail as "legitimate" or as "spam".
5)Regression – attempts to find a function which models the data with the least error.
6)Summarization – providing a more compact representation of the data set, including visualization and report generation.
In business, data mining is the analysis of historical business activities, stored as static data in data warehouse databases. The goal is to reveal hidden patterns and trends.
In recent years, data mining has been used widely in the areas of science and engineering, such as bioinformatics, genetics, medicine, education and electrical power engineering.
Data mining of government records – particularly records of the justice system (i.e., courts, prisons) – enables the discovery of systemic human rights violations in connection to generation and publication of invalid or fraudulent legal records by various government agencies.
