Interpretable AI Series (2) White-box models

This post mainly covers chapter 2 of "Interpretable AI".

White-box models are simple and straightforward machine learning models.

To be concrete, the linear regression model assumes that the label (also known as the result variable, or dependent variable) is the linear combination of the input features. So the absolute value of each feature's coefficient represents the share of the influence, including positive and negative. 

For example, a company plans an advertising budget on both TV and newspapers. What's the best ratio of these 2 channels? The answer is: depends on the prediction of the regression model. Let's say the model shows the distribution of annual incomes is: one-third from newspapers and two-thirds from TV, then we should put two-thirds of the budgets on TV, and one-third on newspapers. The main limitation of the linear model is that most relationships between input and output aren't linear. So it can give us a comparatively rough prediction.

The next white-box model is the decision tree (DT for short). It's also a model that complies with human instinct: a series of "if-then-else". Since DT is nonlinear, there're no coefficients that imply the importance of features. However, we can calculate the importance of each feature by the portion of nodes that belongs to this feature in all nodes' importance.

Here comes 2 questions:

1. How to determine a node belongs to which feature? We know that every node in DT has a splitting rule: split on which feature, and split at which value. We say "the node belongs to feature A" if the splitting feature of the node is A.

2. How to calculate the importance of a node? The importance of a node is computed as the decrease of the cost function or impurity measure of the tree without this node. See the formula on page 38.

The last part of this chapter discusses GAM (Generalized Additive Model) and how to interpret it. Here the author introduced "partial dependence plot". By marginalizing rest features, a partial dependence plot visualizes how the independent variable change with the investigating feature changes (the exact meaning of this sentence need to be studied carefully with exact data context, which I didn't do it yet). See figure 2.21 for detailed illustrations.

Comments

Post a Comment

Popular posts from this blog

2023: On the Road

There're so many dramas happened in the past 3 years. People say there'll be more in the coming 2023. Basically, I agree with this. But no matter what happens in the physical world, we need to concentrate on something really important and interesting. In the past 2022, my most profound change was, that I started thinking seriously about the future living style. My family and friends mostly live like plants, fixed by our roots, and change ourselves to adapt to the environment. We are where we are, proud of its history and culture, and contained by it. It'll not be the case anymore. In 2022, I learned, read, listened, wrote, and spoke in second languages, especially English and Japanese. They bring me a new world. I also read several books on Chinese and middle east histories. The world before me is fusing as a whole sphere. China and Chinese people are a part, and the most familiar part for me, the starting point of my exploration. It gives me enough confidence as a foreig...
Read more

Yet another advice to kids

Sharon posted some advice to the youth (although she herself is young). She suggested being independent, with no fear of failure, which is excellent advice. I'd like to give some more "concrete" tips to kids or myself in childhood (if I have the chance to tell that me). First things first, the following suggestions can't make you more competitive than your peers or perform better in exams. or win the girl make you fascinated. They make your everyday life more exciting and fancy even if you're not rich or powerful. They're neither expensive (so you can achieve these goals even if not born into a rich family) nor painful. What you need is to know them, and keep doing them. OK, here we go. 1. Find a sport, enjoy it, and be good at it. Sport is a way to talk with your body, with the physical world, express yourself, and communicate with people. 2. Be honest, especially to yourself. Living honestly in Chinese society looks stupid and clumsy. Never mind. Being hones...
Read more

The Joy of Reading in Natural Light

Nowadays I read books in paperbacks much less than ebooks. Not because I don't like reading paper books, but after reading for so many years on screen, my eyes are very sensitive to the brightness of reading materials. It's very hard for me to read characters when the material is not bright enough. Unfortunately, the light in my living room is not bright enough for me. This morning it was cloudy, while in the afternoon it got clear, and the blue sky and white clouds appeared. So I moved the desk near the window. To my surprise, the characters on paper books are much more clear and more comfortable to read than in artificial light. Maybe such a simple fact is not worthy to mention. But to me, a programmer spent too much time with the keyboard and screen, this reminds me of the memories of childhood, reading a book, or writing homework in the yard. The characters blurred at sunset. That's when we were going to have super. The window of my living room is east-forwarding, so to...
Read more