如何从 scikit-learn 决策树中提取决策规则？
- 2025-01-16 08:38:00
- admin 原创
- 170
问题描述：
我可以从决策树中经过训练的树中提取底层决策规则（或“决策路径”）作为文本列表吗？
类似于：
if A>0.4 then if B<0.2 then if C>0.8 then class='X'
解决方案 1：
我相信这个答案比这里的其他答案更正确：
from sklearn.tree import _tree
def tree_to_code(tree, feature_names):
tree_ = tree.tree_
feature_name = [
feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature
]
print "def tree({}):".format(", ".join(feature_names))
def recurse(node, depth):
indent = " " * depth
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
print "{}if {} <= {}:".format(indent, name, threshold)
recurse(tree_.children_left[node], depth + 1)
print "{}else: # if {} > {}".format(indent, name, threshold)
recurse(tree_.children_right[node], depth + 1)
else:
print "{}return {}".format(indent, tree_.value[node])
recurse(0, 1)
这将打印出一个有效的 Python 函数。以下是一棵树的示例输出，该树试图返回其输入（一个介于 0 和 10 之间的数字）。
def tree(f0):
if f0 <= 6.0:
if f0 <= 1.5:
return [[ 0.]]
else: # if f0 > 1.5
if f0 <= 4.5:
if f0 <= 3.5:
return [[ 3.]]
else: # if f0 > 3.5
return [[ 4.]]
else: # if f0 > 4.5
return [[ 5.]]
else: # if f0 > 6.0
if f0 <= 8.5:
if f0 <= 7.5:
return [[ 7.]]
else: # if f0 > 7.5
return [[ 8.]]
else: # if f0 > 8.5
return [[ 9.]]
以下是我在其他答案中看到的一些绊脚石：
使用
tree_.threshold == -2来决定一个节点是否是叶子不是一个好主意。如果它是一个阈值为 -2 的真实决策节点怎么办？相反，你应该看看tree.feature或tree.children_*。该线路
features = [feature_names[i] for i in tree_.feature]与我的 sklearn 版本崩溃，因为一些值为tree.tree_.feature-2（特别是对于叶节点）。递归函数中不需要有多个 if 语句，只需一个就可以。
解决方案 2：
我创建了自己的函数来从 sklearn 创建的决策树中提取规则：
import pandas as pd
import numpy as np
from sklearn.tree import DecisionTreeClassifier
# dummy data:
df = pd.DataFrame({'col1':[0,1,2,3],'col2':[3,4,5,6],'dv':[0,1,0,1]})
# create decision tree
dt = DecisionTreeClassifier(max_depth=5, min_samples_leaf=1)
dt.fit(df.ix[:,:2], df.dv)
此函数首先从节点（在子数组中用 -1 标识）开始，然后递归查找父节点。我将其称为节点的“谱系”。在此过程中，我获取创建 if/then/else SAS 逻辑所需的值：
def get_lineage(tree, feature_names):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
features = [feature_names[i] for i in tree.tree_.feature]
# get ids of child nodes
idx = np.argwhere(left == -1)[:,0]
def recurse(left, right, child, lineage=None):
if lineage is None:
lineage = [child]
if child in left:
parent = np.where(left == child)[0].item()
split = 'l'
else:
parent = np.where(right == child)[0].item()
split = 'r'
lineage.append((parent, split, threshold[parent], features[parent]))
if parent == 0:
lineage.reverse()
return lineage
else:
return recurse(left, right, parent, lineage)
for child in idx:
for node in recurse(left, right, child):
print node
下面的元组集合包含我创建 SAS if/then/else 语句所需的一切。我不喜欢do在 SAS 中使用块，这就是我创建描述节点整个路径的逻辑的原因。元组后面的单个整数是路径中终端节点的 ID。所有前面的元组组合在一起以创建该节点。
In [1]: get_lineage(dt, df.columns)
(0, 'l', 0.5, 'col1')
1
(0, 'r', 0.5, 'col1')
(2, 'l', 4.5, 'col2')
3
(0, 'r', 0.5, 'col1')
(2, 'r', 4.5, 'col2')
(4, 'l', 2.5, 'col1')
5
(0, 'r', 0.5, 'col1')
(2, 'r', 4.5, 'col2')
(4, 'r', 2.5, 'col1')
6
解决方案 3：
Scikit learn 在 0.21 版（2019 年 5 月）中引入了一种新方法，export_text用于从树中提取规则。文档在这里。不再需要创建自定义函数。
一旦你已经拟合了模型，你只需要两行代码。首先，导入export_text：
from sklearn.tree import export_text
其次，创建一个包含规则的对象。为了使规则看起来更具可读性，请使用feature_names参数并传递特征名称列表。例如，如果您的模型被调用model并且您的特征在名为的数据框中命名X_train，则可以创建一个名为的对象tree_rules：
tree_rules = export_text(model, feature_names=list(X_train.columns))
然后只需打印或保存tree_rules。您的输出将如下所示：
|--- Age <= 0.63
| |--- EstimatedSalary <= 0.61
| | |--- Age <= -0.16
| | | |--- class: 0
| | |--- Age > -0.16
| | | |--- EstimatedSalary <= -0.06
| | | | |--- class: 0
| | | |--- EstimatedSalary > -0.06
| | | | |--- EstimatedSalary <= 0.40
| | | | | |--- EstimatedSalary <= 0.03
| | | | | | |--- class: 1
解决方案 4：
我修改了Zelazny7提交的代码，打印了一些伪代码：
def get_code(tree, feature_names):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
features = [feature_names[i] for i in tree.tree_.feature]
value = tree.tree_.value
def recurse(left, right, threshold, features, node):
if (threshold[node] != -2):
print "if ( " + features[node] + " <= " + str(threshold[node]) + " ) {"
if left[node] != -1:
recurse (left, right, threshold, features,left[node])
print "} else {"
if right[node] != -1:
recurse (left, right, threshold, features,right[node])
print "}"
else:
print "return " + str(value[node])
recurse(left, right, threshold, features, 0)
如果你调用get_code(dt, df.columns)同一个例子，你将获得：
if ( col1 <= 0.5 ) {
return [[ 1. 0.]]
} else {
if ( col2 <= 4.5 ) {
return [[ 0. 1.]]
} else {
if ( col1 <= 2.5 ) {
return [[ 1. 0.]]
} else {
return [[ 0. 1.]]
}
}
}
解决方案 5：
0.18.0版本中有一个新DecisionTreeClassifier方法。开发人员提供了详尽的（有据可查的）演练。decision_path
演示中打印树结构的第一部分代码似乎没问题。但是，我修改了第二部分的代码来查询一个样本。我的更改用# <--
编辑在拉取请求#8653和#10951# <--中指出错误后，以下代码中标记为 的更改已在演练链接中更新。现在跟进起来容易多了。
sample_id = 0
node_index = node_indicator.indices[node_indicator.indptr[sample_id]:
node_indicator.indptr[sample_id + 1]]
print('Rules used to predict sample %s: ' % sample_id)
for node_id in node_index:
if leave_id[sample_id] == node_id: # <-- changed != to ==
#continue # <-- comment out
print("leaf node {} reached, no decision here".format(leave_id[sample_id])) # <--
else: # < -- added else to iterate through decision nodes
if (X_test[sample_id, feature[node_id]] <= threshold[node_id]):
threshold_sign = "<="
else:
threshold_sign = ">"
print("decision id node %s : (X[%s, %s] (= %s) %s %s)"
% (node_id,
sample_id,
feature[node_id],
X_test[sample_id, feature[node_id]], # <-- changed i to sample_id
threshold_sign,
threshold[node_id]))
Rules used to predict sample 0:
decision id node 0 : (X[0, 3] (= 2.4) > 0.800000011921)
decision id node 2 : (X[0, 2] (= 5.1) > 4.94999980927)
leaf node 4 reached, no decision here
更改sample_id以查看其他示例的决策路径。我没有向开发人员询问这些更改，只是在处理示例时似乎更直观。
解决方案 6：
from StringIO import StringIO
out = StringIO()
out = tree.export_graphviz(clf, out_file=out)
print out.getvalue()
您可以看到一个有向图树。然后，clf.tree_.feature和分别是节点分裂特征数组和节点值数组。您可以从此github 源clf.tree_.value中参考更多详细信息。
解决方案 7：
我需要一种更人性化的决策树规则格式。我正在构建开源AutoML Python 包，很多时候 MLJAR 用户都希望看到树中的确切规则。
这就是我根据paulkernfeld答案实现功能的原因。
def get_rules(tree, feature_names, class_names):
tree_ = tree.tree_
feature_name = [
feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature
]
paths = []
path = []
def recurse(node, path, paths):
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
p1, p2 = list(path), list(path)
p1 += [f"({name} <= {np.round(threshold, 3)})"]
recurse(tree_.children_left[node], p1, paths)
p2 += [f"({name} > {np.round(threshold, 3)})"]
recurse(tree_.children_right[node], p2, paths)
else:
path += [(tree_.value[node], tree_.n_node_samples[node])]
paths += [path]
recurse(0, path, paths)
# sort by samples count
samples_count = [p[-1][1] for p in paths]
ii = list(np.argsort(samples_count))
paths = [paths[i] for i in reversed(ii)]
rules = []
for path in paths:
rule = "if "
for p in path[:-1]:
if rule != "if ":
rule += " and "
rule += str(p)
rule += " then "
if class_names is None:
rule += "response: "+str(np.round(path[-1][0][0][0],3))
else:
classes = path[-1][0][0]
l = np.argmax(classes)
rule += f"class: {class_names[l]} (proba: {np.round(100.0*classes[l]/np.sum(classes),2)}%)"
rule += f" | based on {path[-1][1]:,} samples"
rules += [rule]
return rules
规则按分配给每个规则的训练样本数量排序。对于每个规则，都有关于分类任务的预测类名和预测概率的信息。对于回归任务，仅打印有关预测值的信息。
例子
from sklearn import datasets
from sklearn.tree import DecisionTreeRegressor
from sklearn import tree
from sklearn.tree import _tree
# Prepare the data data
boston = datasets.load_boston()
X = boston.data
y = boston.target
# Fit the regressor, set max_depth = 3
regr = DecisionTreeRegressor(max_depth=3, random_state=1234)
model = regr.fit(X, y)
# Print rules
rules = get_rules(regr, boston.feature_names, None)
for r in rules:
print(r)
印刷规则：
if (RM <= 6.941) and (LSTAT <= 14.4) and (DIS > 1.385) then response: 22.905 | based on 250 samples
if (RM <= 6.941) and (LSTAT > 14.4) and (CRIM <= 6.992) then response: 17.138 | based on 101 samples
if (RM <= 6.941) and (LSTAT > 14.4) and (CRIM > 6.992) then response: 11.978 | based on 74 samples
if (RM > 6.941) and (RM <= 7.437) and (NOX <= 0.659) then response: 33.349 | based on 43 samples
if (RM > 6.941) and (RM > 7.437) and (PTRATIO <= 19.65) then response: 45.897 | based on 29 samples
if (RM <= 6.941) and (LSTAT <= 14.4) and (DIS <= 1.385) then response: 45.58 | based on 5 samples
if (RM > 6.941) and (RM <= 7.437) and (NOX > 0.659) then response: 14.4 | based on 3 samples
if (RM > 6.941) and (RM > 7.437) and (PTRATIO > 19.65) then response: 21.9 | based on 1 samples
我在我的文章中总结了从决策树中提取规则的方法：使用 Scikit-Learn 和 Python 以 3 种方法从决策树中提取规则。
解决方案 8：
现在您可以使用export_text了。
from sklearn.tree import export_text
r = export_text(loan_tree, feature_names=(list(X_train.columns)))
print(r)
来自 sklearn 的完整示例
from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier
from sklearn.tree import export_text
iris = load_iris()
X = iris['data']
y = iris['target']
decision_tree = DecisionTreeClassifier(random_state=0, max_depth=2)
decision_tree = decision_tree.fit(X, y)
r = export_text(decision_tree, feature_names=iris['feature_names'])
print(r)
解决方案 9：
这是您需要的代码
我已经修改了最受欢迎的代码，以便在 jupyter 笔记本 python 3 中正确缩进
import numpy as np
from sklearn.tree import _tree
def tree_to_code(tree, feature_names):
tree_ = tree.tree_
feature_name = [feature_names[i]
if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature]
print("def tree({}):".format(", ".join(feature_names)))
def recurse(node, depth):
indent = " " * depth
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
print("{}if {} <= {}:".format(indent, name, threshold))
recurse(tree_.children_left[node], depth + 1)
print("{}else: # if {} > {}".format(indent, name, threshold))
recurse(tree_.children_right[node], depth + 1)
else:
print("{}return {}".format(indent, np.argmax(tree_.value[node])))
recurse(0, 1)
解决方案 10：
因为大家都很热心，所以我将对 Zelazny7 和 Daniele 的漂亮解决方案进行修改。这个适用于 Python 2.7，带有制表符，使其更具可读性：
def get_code(tree, feature_names, tabdepth=0):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
features = [feature_names[i] for i in tree.tree_.feature]
value = tree.tree_.value
def recurse(left, right, threshold, features, node, tabdepth=0):
if (threshold[node] != -2):
print ' ' * tabdepth,
print "if ( " + features[node] + " <= " + str(threshold[node]) + " ) {"
if left[node] != -1:
recurse (left, right, threshold, features,left[node], tabdepth+1)
print ' ' * tabdepth,
print "} else {"
if right[node] != -1:
recurse (left, right, threshold, features,right[node], tabdepth+1)
print ' ' * tabdepth,
print "}"
else:
print ' ' * tabdepth,
print "return " + str(value[node])
recurse(left, right, threshold, features, 0)
解决方案 11：
我一直在经历这个问题，但我需要以这种格式编写规则
if A>0.4 then if B<0.2 then if C>0.8 then class='X'
因此我改编了@paulkernfeld 的答案（谢谢），您可以根据自己的需要进行自定义
def tree_to_code(tree, feature_names, Y):
tree_ = tree.tree_
feature_name = [
feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature
]
pathto=dict()
global k
k = 0
def recurse(node, depth, parent):
global k
indent = " " * depth
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
s= "{} <= {} ".format( name, threshold, node )
if node == 0:
pathto[node]=s
else:
pathto[node]=pathto[parent]+' & ' +s
recurse(tree_.children_left[node], depth + 1, node)
s="{} > {}".format( name, threshold)
if node == 0:
pathto[node]=s
else:
pathto[node]=pathto[parent]+' & ' +s
recurse(tree_.children_right[node], depth + 1, node)
else:
k=k+1
print(k,')',pathto[parent], tree_.value[node])
recurse(0, 1, 0)
解决方案 12：
这里提供一种使用SKompiler库将整棵树转换成单个（不一定太易于人类阅读的）python 表达式的方法：
from skompiler import skompile
skompile(dtree.predict).to('python/code')
解决方案 13：
这是基于@paulkernfeld 的答案。如果您有一个包含特征的数据框 X 和一个包含响应的目标数据框 y，并且您想知道哪个 y 值在哪个节点结束（并且还想相应地绘制它），您可以执行以下操作：
def tree_to_code(tree, feature_names):
from sklearn.tree import _tree
codelines = []
codelines.append('def get_cat(X_tmp):
')
codelines.append(' catout = []
')
codelines.append(' for codelines in range(0,X_tmp.shape[0]):
')
codelines.append(' Xin = X_tmp.iloc[codelines]
')
tree_ = tree.tree_
feature_name = [
feature_names[i] if i != _tree.TREE_UNDEFINED else "undefined!"
for i in tree_.feature
]
#print "def tree({}):".format(", ".join(feature_names))
def recurse(node, depth):
indent = " " * depth
if tree_.feature[node] != _tree.TREE_UNDEFINED:
name = feature_name[node]
threshold = tree_.threshold[node]
codelines.append ('{}if Xin["{}"] <= {}:
'.format(indent, name, threshold))
recurse(tree_.children_left[node], depth + 1)
codelines.append( '{}else: # if Xin["{}"] > {}
'.format(indent, name, threshold))
recurse(tree_.children_right[node], depth + 1)
else:
codelines.append( '{}mycat = {}
'.format(indent, node))
recurse(0, 1)
codelines.append(' catout.append(mycat)
')
codelines.append(' return pd.DataFrame(catout,index=X_tmp.index,columns=["category"])
')
codelines.append('node_ids = get_cat(X)
')
return codelines
mycode = tree_to_code(clf,X.columns.values)
# now execute the function and obtain the dataframe with all nodes
exec(''.join(mycode))
node_ids = [int(x[0]) for x in node_ids.values]
node_ids2 = pd.DataFrame(node_ids)
print('make plot')
import matplotlib.cm as cm
colors = cm.rainbow(np.linspace(0, 1, 1+max( list(set(node_ids)))))
#plt.figure(figsize=cm2inch(24, 21))
for i in list(set(node_ids)):
plt.plot(y[node_ids2.values==i],'o',color=colors[i], label=str(i))
mytitle = ['y colored by node']
plt.title(mytitle ,fontsize=14)
plt.xlabel('my xlabel')
plt.ylabel(tagname)
plt.xticks(rotation=70)
plt.legend(loc='upper center', bbox_to_anchor=(0.5, 1.00), shadow=True, ncol=9)
plt.tight_layout()
plt.show()
plt.close
虽然不是最优雅的版本，但它可以完成工作......
解决方案 14：
下面的代码是我在 anaconda python 2.7 下加上一个名为“pydot-ng”的包来制作带有决策规则的 PDF 文件的方法：
from sklearn import tree
clf = tree.DecisionTreeClassifier(max_leaf_nodes=n)
clf_ = clf.fit(X, data_y)
feature_names = X.columns
class_name = clf_.classes_.astype(int).astype(str)
def output_pdf(clf_, name):
from sklearn import tree
from sklearn.externals.six import StringIO
import pydot_ng as pydot
dot_data = StringIO()
tree.export_graphviz(clf_, out_file=dot_data,
feature_names=feature_names,
class_names=class_name,
filled=True, rounded=True,
special_characters=True,
node_ids=1,)
graph = pydot.graph_from_dot_data(dot_data.getvalue())
graph.write_pdf("%s.pdf"%name)
output_pdf(clf_, name='filename%s'%n)
这里展示树木摄影作品
解决方案 15：
这是一个函数，在 python 3 下打印 scikit-learn 决策树的规则，并带有条件块的偏移量，以使结构更具可读性：
def print_decision_tree(tree, feature_names=None, offset_unit=' '):
'''Plots textual representation of rules of a decision tree
tree: scikit-learn representation of tree
feature_names: list of feature names. They are set to f1,f2,f3,... if not specified
offset_unit: a string of offset of the conditional block'''
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
value = tree.tree_.value
if feature_names is None:
features = ['f%d'%i for i in tree.tree_.feature]
else:
features = [feature_names[i] for i in tree.tree_.feature]
def recurse(left, right, threshold, features, node, depth=0):
offset = offset_unit*depth
if (threshold[node] != -2):
print(offset+"if ( " + features[node] + " <= " + str(threshold[node]) + " ) {")
if left[node] != -1:
recurse (left, right, threshold, features,left[node],depth+1)
print(offset+"} else {")
if right[node] != -1:
recurse (left, right, threshold, features,right[node],depth+1)
print(offset+"}")
else:
print(offset+"return " + str(value[node]))
recurse(left, right, threshold, features, 0,0)
解决方案 16：
您还可以通过区分它所属的类别甚至提及它的输出值，使其更具信息量。
def print_decision_tree(tree, feature_names, offset_unit=' '):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
value = tree.tree_.value
if feature_names is None:
features = ['f%d'%i for i in tree.tree_.feature]
else:
features = [feature_names[i] for i in tree.tree_.feature]
def recurse(left, right, threshold, features, node, depth=0):
offset = offset_unit*depth
if (threshold[node] != -2):
print(offset+"if ( " + features[node] + " <= " + str(threshold[node]) + " ) {")
if left[node] != -1:
recurse (left, right, threshold, features,left[node],depth+1)
print(offset+"} else {")
if right[node] != -1:
recurse (left, right, threshold, features,right[node],depth+1)
print(offset+"}")
else:
#print(offset,value[node])
#To remove values from node
temp=str(value[node])
mid=len(temp)//2
tempx=[]
tempy=[]
cnt=0
for i in temp:
if cnt<=mid:
tempx.append(i)
cnt+=1
else:
tempy.append(i)
cnt+=1
val_yes=[]
val_no=[]
res=[]
for j in tempx:
if j=="[" or j=="]" or j=="." or j==" ":
res.append(j)
else:
val_no.append(j)
for j in tempy:
if j=="[" or j=="]" or j=="." or j==" ":
res.append(j)
else:
val_yes.append(j)
val_yes = int("".join(map(str, val_yes)))
val_no = int("".join(map(str, val_no)))
if val_yes>val_no:
print(offset,'[1m',"YES")
print('[0m')
elif val_no>val_yes:
print(offset,'[1m',"NO")
print('[0m')
else:
print(offset,'[1m',"Tie")
print('[0m')
recurse(left, right, threshold, features, 0,0)
解决方案 17：
这是我以可以直接在 SQL 中使用的形式提取决策规则的方法，因此可以按节点对数据进行分组。（基于以前发帖人的方法。）
结果将是CASE可以复制到 SQL 语句的后续子句，例如
`SELECT COALESCE(CASE WHEN <conditions> THEN > <NodeA>, > *CASE WHEN
<conditions> THEN <NodeB>, > ....)NodeName, > FROM <table or view>`
import numpy as np
import pickle
feature_names=.............
features = [feature_names[i] for i in range(len(feature_names))]
clf= pickle.loads(trained_model)
impurity=clf.tree_.impurity
importances = clf.feature_importances_
SqlOut=""
#global Conts
global ContsNode
global Path
#Conts=[]#
ContsNode=[]
Path=[]
global Results
Results=[]
def print_decision_tree(tree, feature_names, offset_unit='' ''):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
value = tree.tree_.value
if feature_names is None:
features = [''f%d''%i for i in tree.tree_.feature]
else:
features = [feature_names[i] for i in tree.tree_.feature]
def recurse(left, right, threshold, features, node, depth=0,ParentNode=0,IsElse=0):
global Conts
global ContsNode
global Path
global Results
global LeftParents
LeftParents=[]
global RightParents
RightParents=[]
for i in range(len(left)): # This is just to tell you how to create a list.
LeftParents.append(-1)
RightParents.append(-1)
ContsNode.append("")
Path.append("")
for i in range(len(left)): # i is node
if (left[i]==-1 and right[i]==-1):
if LeftParents[i]>=0:
if Path[LeftParents[i]]>" ":
Path[i]=Path[LeftParents[i]]+" AND " +ContsNode[LeftParents[i]]
else:
Path[i]=ContsNode[LeftParents[i]]
if RightParents[i]>=0:
if Path[RightParents[i]]>" ":
Path[i]=Path[RightParents[i]]+" AND not " +ContsNode[RightParents[i]]
else:
Path[i]=" not " +ContsNode[RightParents[i]]
Results.append(" case when " +Path[i]+" then ''" +"{:4d}".format(i)+ " "+"{:2.2f}".format(impurity[i])+" "+Path[i][0:180]+"''")
else:
if LeftParents[i]>=0:
if Path[LeftParents[i]]>" ":
Path[i]=Path[LeftParents[i]]+" AND " +ContsNode[LeftParents[i]]
else:
Path[i]=ContsNode[LeftParents[i]]
if RightParents[i]>=0:
if Path[RightParents[i]]>" ":
Path[i]=Path[RightParents[i]]+" AND not " +ContsNode[RightParents[i]]
else:
Path[i]=" not "+ContsNode[RightParents[i]]
if (left[i]!=-1):
LeftParents[left[i]]=i
if (right[i]!=-1):
RightParents[right[i]]=i
ContsNode[i]= "( "+ features[i] + " <= " + str(threshold[i]) + " ) "
recurse(left, right, threshold, features, 0,0,0,0)
print_decision_tree(clf,features)
SqlOut=""
for i in range(len(Results)):
SqlOut=SqlOut+Results[i]+ " end,"+chr(13)+chr(10)
解决方案 18：
修改了Zelazny7的代码以从决策树中获取SQL。
# SQL from decision tree
def get_lineage(tree, feature_names):
left = tree.tree_.children_left
right = tree.tree_.children_right
threshold = tree.tree_.threshold
features = [feature_names[i] for i in tree.tree_.feature]
le='<='
g ='>'
# get ids of child nodes
idx = np.argwhere(left == -1)[:,0]
def recurse(left, right, child, lineage=None):
if lineage is None:
lineage = [child]
if child in left:
parent = np.where(left == child)[0].item()
split = 'l'
else:
parent = np.where(right == child)[0].item()
split = 'r'
lineage.append((parent, split, threshold[parent], features[parent]))
if parent == 0:
lineage.reverse()
return lineage
else:
return recurse(left, right, parent, lineage)
print 'case '
for j,child in enumerate(idx):
clause=' when '
for node in recurse(left, right, child):
if len(str(node))<3:
continue
i=node
if i[1]=='l': sign=le
else: sign=g
clause=clause+i[3]+sign+str(i[2])+' and '
clause=clause[:-4]+' then '+str(j)
print clause
print 'else 99 end as clusters'
解决方案 19：
这是一个通过转换输出从决策树生成 Python 代码的函数export_text：
import string
from sklearn.tree import export_text
def export_py_code(tree, feature_names, max_depth=100, spacing=4):
if spacing < 2:
raise ValueError('spacing must be > 1')
# Clean up feature names (for correctness)
nums = string.digits
alnums = string.ascii_letters + nums
clean = lambda s: ''.join(c if c in alnums else '_' for c in s)
features = [clean(x) for x in feature_names]
features = ['_'+x if x[0] in nums else x for x in features if x]
if len(set(features)) != len(feature_names):
raise ValueError('invalid feature names')
# First: export tree to text
res = export_text(tree, feature_names=features,
max_depth=max_depth,
decimals=6,
spacing=spacing-1)
# Second: generate Python code from the text
skip, dash = ' '*spacing, '-'*(spacing-1)
code = 'def decision_tree({}):
'.format(', '.join(features))
for line in repr(tree).split('
'):
code += skip + "# " + line + '
'
for line in res.split('
'):
line = line.rstrip().replace('|',' ')
if '<' in line or '>' in line:
line, val = line.rsplit(maxsplit=1)
line = line.replace(' ' + dash, 'if')
line = '{} {:g}:'.format(line, float(val))
else:
line = line.replace(' {} class:'.format(dash), 'return')
code += skip + line + '
'
return code
使用示例：
res = export_py_code(tree, feature_names=names, spacing=4)
print (res)
示例输出：
def decision_tree(f1, f2, f3):
# DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=3,
# max_features=None, max_leaf_nodes=None,
# min_impurity_decrease=0.0, min_impurity_split=None,
# min_samples_leaf=1, min_samples_split=2,
# min_weight_fraction_leaf=0.0, presort=False,
# random_state=42, splitter='best')
if f1 <= 12.5:
if f2 <= 17.5:
if f1 <= 10.5:
return 2
if f1 > 10.5:
return 3
if f2 > 17.5:
if f2 <= 22.5:
return 1
if f2 > 22.5:
return 1
if f1 > 12.5:
if f1 <= 17.5:
if f3 <= 23.5:
return 2
if f3 > 23.5:
return 3
if f1 > 17.5:
if f1 <= 25:
return 1
if f1 > 25:
return 2
上面的例子是用 生成的names = ['f'+str(j+1) for j in range(NUM_FEATURES)]。
一个方便的功能是它可以生成更小的文件大小和更小的间距。只需设置spacing=2。
解决方案 20：
在 .txt 文件中编写规则
from sklearn.tree import export_text
r = export_text(clf, feature_names=feature_names)
f = open("Rules_set.txt", "w")
f.write(r)
文本文件
从文件读取规则
file1 = open("Rules_set.txt","r")
data = file1.readlines()
dic = {}
first = None
for line in data:
if( 'class' in line):
#print(line.index('class'))
rule = ' and '.join(list(dic.values()))
rule = rule + ' ' + line[line.index('class'):]
print(rule.strip())
else:
for char in line:
if char.isalpha():
index = line.index(char)
if first == None:
first = index
if first == index:
dic = {}
dic[index] = f'({line[index:].strip()})'
break
规则
解决方案 21：
显然很久以前就有人决定尝试将以下函数添加到官方 scikit 的树导出函数中（基本上只支持 export_graphviz）
def export_dict(tree, feature_names=None, max_depth=None) :
"""Export a decision tree in dict format.
以下是他的完整承诺：
不太清楚这条评论发生了什么。但你也可以尝试使用该功能。
我认为这值得向 scikit-learn 的优秀人员提出严肃的文档请求，以正确记录sklearn.tree.TreeAPI，即DecisionTreeClassifier以其属性公开的底层树结构tree_。
解决方案 22：
只需像这样使用sklearn.tree中的函数
from sklearn.tree import export_graphviz
export_graphviz(tree,
out_file = "tree.dot",
feature_names = tree.columns) //or just ["petal length", "petal width"]
然后在项目文件夹中查找文件tree.dot，复制所有内容并将其粘贴到这里http://www.webgraphviz.com/并生成您的图表:)
解决方案 23：
感谢@paulkerfeld 的精彩解决方案。除了他的解决方案之外，对于所有想要拥有树的序列化版本的人，只需使用tree.threshold、、和。由于叶子没有分裂，因此没有特征名称和子元素，它们在和中的占位符是和。每个分裂都由分配一个唯一索引。
请注意，tree.children_left形状为tree.children_right`tree.featuretree.valuetree.featuretree.children_***_tree.TREE_UNDEFINED_tree.TREE_LEAFdepth first search` tree.value`[n, 1, 1]`
解决方案 24：
从这个答案中，您可以获得一个可读且高效的表示：https://stackoverflow.com/a/65939892/3746632
输出如下所示。X 是一维向量，表示单个实例的特征。
from numba import jit,njit
@njit
def predict(X):
ret = 0
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
ret += 1
else: # if w_pizza > 0.5
pass
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
ret += 1
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
ret += 1
else: # if w_mexico > 0.5
ret += 1
else: # if w_pizza > 0.5
pass
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
ret += 1
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
ret += 1
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
pass
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
ret += 1
else: # if w_pizza > 0.5
ret += 1
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
ret += 1
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
pass
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
pass
if X[0] <= 0.5: # if w_pizza <= 0.5
if X[1] <= 0.5: # if w_mexico <= 0.5
if X[2] <= 0.5: # if w_reusable <= 0.5
ret += 1
else: # if w_reusable > 0.5
pass
else: # if w_mexico > 0.5
pass
else: # if w_pizza > 0.5
pass
return ret/10
解决方案 25：
当面临同样的问题时，我想要一种从纯文本中提取规则的方法，然后不一定将结果与“if”语句绑定，而只将结果存储在列表中。
考虑到我无法安装某些软件包，我想出了使用“纯 Python”的解决方案。这里的缺点是需要知道树的深度，如果深度不固定，它可能不会扩展。比我聪明的人可能会通过将变量更改max_depth为自适应变量来找到答案，该变量每次考虑标记时都会变大，但不包含“值”作为子字符串。
from typing import List, Dict
def process_tree(tree_text: str, max_depth: int = 4) -> List[List[str]]:
lines = tree_text.strip().split('
')
rules: List[List[str]] = []
stack: List[str] = []
num_appearence: Dict[str, int] = {}
for line in lines:
level: int = line.count('|')
token: str = line.split('|---')[-1]
if token not in num_appearence.keys() and 'value' not in token:
num_appearence[token] = int(2**(max_depth-level))
if 'value' not in token:
stack.append(token)
continue
rules.append(stack)
for element in stack:
num_appearence[element] -= 1
stack = [element for element in stack if num_appearence[element] > 0]
return rules
我知道如果预先分配一些内存并设置一些条件，效果会更好yield，但目前这个最适合我。以下是示例：
输入
tree_text = """|--- var1 <= c1
| |--- var2 <= c2
| | |--- var3 <= c3
| | | |--- var4 <= c4
| | | | |--- value: x1
| | | |--- var5 <= c5
| | | | |--- value: x2
| | |--- var6 > c6
| | | |--- var7 > c7
| | | | |--- value: x3
| | | |--- var8 <= c8
| | | | |--- value: x4
| |--- var9 > c9
| | |--- var10 <= c10
| | | |--- var11 <= c11
| | | | |--- value: x5
| | | |--- var12 > c12
| | | | |--- value: x6
| | |--- var13 > c13
| | | |--- var14 <= c14
| | | | |--- value: x7
| | | |--- var15 > c15
| | | | |--- value: x8
|--- var16 > c16
| |--- var17 <= c17
| | |--- var18 <= c18
| | | |--- var19 <= c19
| | | | |--- value: x9
| | | |--- var20 > c20
| | | | |--- value: x10
| | |--- var21 > c21
| | | |--- var22 <= c22
| | | | |--- value: x11
| | | |--- var23 > c23
| | | | |--- value: x12
"""
rules = process_tree(tree_text)
for i, rule in enumerate(rules):
print(f'Rule {i}: {rule}')
输出
Rule 0: [' var1 <= c1', ' var2 <= c2', ' var3 <= c3', ' var4 <= c4']
Rule 1: [' var1 <= c1', ' var2 <= c2', ' var3 <= c3', ' var5 <= c5']
Rule 2: [' var1 <= c1', ' var2 <= c2', ' var6 > c6', ' var7 > c7']
Rule 3: [' var1 <= c1', ' var2 <= c2', ' var6 > c6', ' var8 <= c8']
Rule 4: [' var1 <= c1', ' var9 > c9', ' var10 <= c10', ' var11 <= c11']
Rule 5: [' var1 <= c1', ' var9 > c9', ' var10 <= c10', ' var12 > c12']
Rule 6: [' var1 <= c1', ' var9 > c9', ' var13 > c13', ' var14 <= c14']
Rule 7: [' var1 <= c1', ' var9 > c9', ' var13 > c13', ' var15 > c15']
Rule 8: [' var16 > c16', ' var17 <= c17', ' var18 <= c18', ' var19 <= c19']
Rule 9: [' var16 > c16', ' var17 <= c17', ' var18 <= c18', ' var20 > c20']
Rule 10: [' var16 > c16', ' var17 <= c17', ' var21 > c21', ' var22 <= c22']
Rule 11: [' var16 > c16', ' var17 <= c17', ' var21 > c21', ' var23 > c23']
相关推荐
热门文章
项目管理软件有哪些？
热门标签
曾咪二维码
扫码咨询，免费领取项目管理大礼包！
云禅道AD
