Decision Tree Algorithm in Data Mining

A decision tree is a computer-intensive statistical method for categorizing items including such things as people, businesses, websites or anything else based upon variables. It is especially useful when there are a large number of variables, which may interact statistically.

Growing a Tree

A decision tree begins with all the subjects in one group. The program then goes through every possible division of each variable, to find the best way to split the node into two nodes. Each of those nodes is then split again and so on, until the criteria for stopping are met. These criteria can be set by the user and includes such things as size of a node, purity of a node, size of child nodes and other criteria.

Pruning a Tree

Research shows that it is better to grow a large tree and then prune it. Various methods of pruning exist, but all are based on the idea of getting a tree that is stable; one which works not just with the sample you have, but on other samples as well.

Cross-validating a Tree

Ideally, you will have enough data to grow and prune a tree on part of your data and then test it on another part of your data. If this is not possible, there are other methods of cross-validating trees.

Benefits

One benefit to decision trees is that the output is easy to explain to people without statistical training. Another benefit is that they allow you to look at interactions that occur in only some parts of the data.

References

• "Recursive Partitioning and Applications" Zhang and Singer, 2010