[Notes] CSCI 585 discussion 8 Data Mining

Why do we need ‘mine’ data in order to extract value from it?

• data did not give obvious insight, need algorithm to extract unknown pattern

Demystifying data mining - a whitepaper from SAS. Here are details on SAS Enterprise Miner, the tool featured in the whitepaper.

• data preparation
• exploratory data
• variable transformation
• model building
• testing and validation
• put models into production
• monitoring the model performance

This is an intro’ (to DM). Also check out this followup, tutorial and list of links.

DM examples from Egon.

Examples from Matillion.

An episode from The Colbert Report. Watch the first 4 minutes, then read this.

An assortment of related (to DM) concepts, algorithms….

time series analysis GBM CatBoost PCA [Principal Components Analysis] (eg. look at this) LDA [Linear Discriminant Analysis] (eg. see this) Sammon Projection Ridge regression …