490 questions
3
votes
1
answer
162
views
Parallelization with "future" always uses all cores
I am following the documentation of the shapr package (version 1.0.5) to use parallelization, but whatever number of "workers" I specify, future always uses all available processor cores.
...
0
votes
0
answers
50
views
module 'shap' has no attribute 'DeepExplainer' with pytorch
I wrote a code for SHAP on images. It was working for two days, then it brought following error :
module 'shap' has no attribute 'DeepExplainer'
I uninstalled and installed the SHAP module again and ...
2
votes
2
answers
82
views
sv_dependence plot with shapviz and same feature for x axis and colour
I am sorry to ask this question again (first attempt here: shapvix sv_dependence color_var based on same feature), but it was closed because not enough details were provided. Below is an updated ...
2
votes
1
answer
118
views
Why shap's explainer.model.predict() and model.predict don't match?
I have a machine learning model and I calculated SHAP on it using following code:
import shap
background = shap.kmeans(X_dev, k=100)
explainer = shap.TreeExplainer(model, feature_perturbation="...
0
votes
0
answers
69
views
SHAP DimensionError: Length of features is not equal to the length of shap_values when using TreeExplainer with RandomForest
I'm encountering an issue when applying SHAP (SHapley Additive exPlanations) to my model predictions. Specifically, when I use shap.TreeExplainer with a RandomForestClassifier, I get the following ...
0
votes
0
answers
117
views
SHAP value in TreeExplainer: Additivity check failed in TreeExplainer
I am trying to obtain the SHAP values of a Random Forest model for binary classification, trained in Python. I am using the following code:
final_model = RandomForestClassifier(random_state=42, **...
-2
votes
1
answer
202
views
Can SHAP be used to calculate Shapley values for traditional coalitional game theory applications?
I am currently working on a project that needs me to calculate the Shapley values of players that enter into a coalition. I've written up the value function, but was wondering if there was a way to ...
1
vote
2
answers
256
views
Shap text plot does not show properly in notebook
I am running the demo code provided here. But I do not get the same plots with the nice coloring and highlights.
Here is how it looks like in the source notebook:
Here is how mine looks like
I am ...
0
votes
0
answers
29
views
Shap plots on a gnn model and specifically SGC model
I'm having a lot issues implementing shap plot for my SGC model.
Any one experienced the problem the same problem ,can give an example.
The problem is that there are a lot errors when try shap with ...
0
votes
1
answer
382
views
Length of features is not equal to the length of SHAP Values
Im running a random forest model and to get some feature importance and Im trying to run a SHAP analysis. The problem is that every time I try to plot the shap values, I keep getting this error:
...
0
votes
1
answer
171
views
UnicodeDecodeError when using SHAP 0.42 and xgboost 2.1.1
I am trying to explain my xgboost (v 2.1.1) model with shap (v 0.42) but I get this error:
UnicodeDecodeError Traceback (most recent call last)
Cell In[53], line 1
----> 1 ...
0
votes
0
answers
50
views
Problem with Shap values and feature importance plotting with GAN model
I have written this code to measure SHAP values and plot feature importance. However, I got an error when plotting the shape values. I used the gen_GRU_model_179.h5 generated model, my model expects a ...
0
votes
0
answers
21
views
Can I perform XAI for an input occluded image with the pretrained model (ArcFace) for detecting occlusion?
I am doing XAI for the input occluded image using a pretrained ArcFace model to find the occluded area of the image and plot the PLQ map.
Here for XAI, I have used shap library in python. I have using ...
1
vote
0
answers
170
views
SHAP plot with categorical columns
Before one-hot encoding, the input data consists of 2 categorical columns (category1, category2). category1 is among A,B,C and category2 is among X,Y. After the one-hot encoding, the input data ...
1
vote
2
answers
120
views
How to use treeshap based on mlr3 framework?
I am attempting to utilize the Treeshap package to calculate SHAP values. However, I encountered an error when trying to apply it to models created using the mlr3 framework. Specifically, I am unsure ...
1
vote
0
answers
315
views
Kernel-stop error occurred during shap-value calculation
I am trying to predict the occurrence of a certain phenomenon using about 9500 persons’ data with 36-37 predictor items using sklearn. It is a binary classification problem (occur or not-occur). ...
1
vote
1
answer
189
views
increase the decimal places of the data labels in a SHAP waterfall plot
Some of the data labels of the SHAP waterfall plot that I get are equal to zero. However, they are not and the plot itself rounds them. Therefore, I need to increase the decimal places of the labels.
...
1
vote
0
answers
95
views
SHAP for transformer-based Regression
I'm developing a transformer-based regression model using a classification model with num labels set to 1 so that it can be used for regression.
from transformers import AutoTokenizer, ...
0
votes
0
answers
93
views
Why has the shap value array in a shap explanation object a different shape than expected from shap plots?
I'm creating an Explanation object using SHAP. This object holds shap values, base values and the actual data. My problem is that the shap value array in the explanation object comes in a shape that ...
0
votes
0
answers
78
views
Python session crash when use SHAP
I have some problems with shap (python) in Google Colab notebook; I want to use ResNet50 with CIFAR100 dataset with Partition explainer; I get a session crash but I don't understand exactly why:
Here ...
2
votes
0
answers
221
views
SHAP Additivity Check Fails with Astronomical SHAP Values for RandomForestClassifier
So I trained my model and here I can share some relevant parts regarding the issue:
import pandas as pd
import pickle
from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.ensemble ...
1
vote
1
answer
402
views
SHAP values for random survival forest
I want to plot the SHAP values for my RSF model; here is the code and error:
xvars <- c("RIDRETH1", "RXDLIPID", "DRXTKCAL", "DRXTPROT", "DRXTCARB", ...
1
vote
1
answer
88
views
Why I get different shap values according to the different order of inputs by using function shap.Explainer?
I trained a two-classification model and wanted to use shap.Explainer to analyze feature contribution.
The code was:
def f(x):
return model.predict_proba(x)[:, 1]
X100 = shap.utils.sample(X_train,...
0
votes
1
answer
187
views
How to make shap.plots.scatter with xgboost.DMatrix holding missing data?
I have a dataset with missing data. They are encoded as NaN. This is fine for model fitting with XGBoost. When I want to understand the model, analyzing model importance with SHAP scatter plots, I am ...
1
vote
0
answers
92
views
How to plot stacked bar chat in the latest version of SHAP
I am trying to reproduce the following example with the latest version 0.46.0 of SHAP open source. However, it seems to be weird that the result is not a bar chat as in the example.
My attempt: Then, ...
2
votes
1
answer
349
views
Python in Excel: How to install other Anaconda packages
Python in Excel includes some packages in the Anaconda distribution. +info, anaconda distribution packages.
However, is it possible to install additional packages that are still in the Anaconda ...
0
votes
0
answers
62
views
How to write Bangla in shap graph
I have used shap on my Bangla dataset and plotted bar graph with following code:
pred = transformers.pipeline("text-classification", model=model, tokenizer=tokenizer, device=0, ...
0
votes
1
answer
286
views
Error when calculating SHAP value in xgboost model - feature names are different?
I have trained an XGBoost model using caret and now, I am calculating the mean SHAP value of each predictor using the package SHAPforxgboost, using the following code:
library(SHAPforxgboost)
...
1
vote
0
answers
58
views
Shap IndexError :index 16 is out of bounds for axis 1 with size 16
import tensorflow as tf
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import os
from datetime import datetime
from sklearn.preprocessing import StandardScaler,MinMaxScaler,...
1
vote
1
answer
427
views
Missing value has a non-zero SHAP value for XGBoost model
I want to represent a dataset with the SHAP values of each entry (I am not interested in the actual data, but rather its importance). To calculate SHAP values I use an XGBoost model.
model = xgboost....
0
votes
1
answer
124
views
I want to colorize the points of a Shapviz dependence output acording to the variable
I have been trying to change the colors of the points on each variable, but even though i've tried using scale_x_discrete, by and changing the colors, or using scale_color_manual(), nothing works for ...
2
votes
1
answer
350
views
SHAP values for linear model different from those calculated manually
I train a linear model to predict house price, and then I compare the Shapley values calculation manually vs the values returned by the SHAP library and they are slightly different.
My understanding ...
0
votes
0
answers
168
views
SHAP for multiclass classification
I have a small dataset with 816 rows and 8 numerical features.
The target variable is categorical, with 4 different categories.
I trained a XGB model on the dataset and after that I try to use SHAP ...
0
votes
0
answers
198
views
Is there a way to get a Shap.kernelExplainer.shap_values in Python to accept 3-dimensional data?
I want to do a SHAP analysis of a model. The model is a RNN that takes a whole timeseries as input, where every time series is in the format (101, 12). However, the SHAP package won't accept 3-...
2
votes
0
answers
87
views
Why the contribution of a categorical value in SHAP trained on Catboost differs from observation to observation
Context
Let's imagine I am interested in predicting sepal length in the iris dataset using catboost.
Objective
My main objective is understanding the effect of each categorical value for target in the ...
0
votes
1
answer
625
views
SHAP value explanations in binary classification [closed]
I was trying to interpret my binary classification model using the SHAP value for each feature. I am wondering:
Does the positive SHAP value mean the feature contributed more to predicting the "1&...
1
vote
1
answer
1k
views
K-means clustering and nsamples for KernelExplainer
I have a dataset which contains roughly 50,000 observations. I want to compute the Shapley value using KernelExplainer after estimating an ElasticNet for regression. Is there any reference or rule ...
0
votes
0
answers
67
views
How does the feature contribution being calculated in the Explainer Dashboard in Python?
I built a soft voting model and wanted to calculate the feature contribution for an individual prediction. However, I found discrepancies in the result between the feature contributions calculated by ...
0
votes
0
answers
76
views
Is there a way of measuring the initial SHAP value of two features that have been used to create a third, using a ratio?
In the context of machine learning on python, regression or classification whatever, I sometimes do the ratio of two features to make a third feature and drop the first two. For example:
...
1
vote
0
answers
106
views
SHAP Summary with PLOTLY
I want to recreate the SHAP Summary Plot with the help of the plotly library.
I found this beautiful post: https://community.plotly.com/t/shap-like-bee-swarm-plots/68015
But I have no idea how the ...
1
vote
1
answer
272
views
How to fit kernelshap and shapviz into a tidymodels workflow?
I am using a mtcars dataset and tune an xgboost model to predict mpg
# Load required packages
library(tidymodels)
library(xgboost)
library(DALEX)
library(DALEXtra)
# Load example data
data(mtcars)
# ...
1
vote
0
answers
230
views
SHAP DeepExplainer gives an error for model with 1D CNN as a first layer
Initially when I tried to work with KernelExplainer, it won't work as dimension of my dataset is (no. of samples, 186, 1). I saw from few forums that KernelExplainer won't work(correct me if i am ...
1
vote
0
answers
46
views
SHAP DeepExplainer Error: 'tuple' object has no attribute 'as_list' 'device' (CNN, LSTM model)
I compiled, fit the CNN LSTM model.
normalized_train_data is my whole dataset, which I have split to training and test set.
I want to see features' contribution by running shpaley test.
The code even ...
0
votes
1
answer
375
views
How to remove "other features" and move the position of E[f(x)] value from SHAP waterfall plot?
I'm using the {shapviz} package and I would like to plot a waterfall plot with only 5 categories, but without the "other features" category (which refers to the sum of the other features). ...
1
vote
1
answer
234
views
Is there a R function to use SHAP with the result of rfsrc
I am using rfsrc in Rstudio to run my bioinformatics data (included survival status and time), and I developed my competing risks model. Although I have generated my result with nested cross ...
0
votes
0
answers
267
views
Compute SHAP Value for a systems of model using shap.DeepExplainer
I have trained a two-stage ML model training for binary classification. In the first stage, I applied an autoencoder model consisting of encoder and decoder. The encoder model has reduced the ...
1
vote
0
answers
292
views
How to use SHAP Values for grouped Feature Importance?
What I do:
I analyse different biomarkers from EEG data with the help of different machine learning algorithms and different pre-processing steps etc. This results in several models for each ...
1
vote
2
answers
294
views
How do I change the dot size in the SHAP beeswarm plot?
I'm using the shap library for python and pretending to use it with a couple of machine learning models. I'm just a beginner in all of this.
I have been trying to represent the beeswarm plot to ...
1
vote
1
answer
632
views
SHAP Waterfall diagram as a matplotlib subplot?
Hi i would like to Show side by side diagrams from SHAP Library:
Waterfall Diagram :API Reference https://shap.readthedocs.io/en/latest/generated/shap.plots.waterfall.html#shap.plots.waterfall
Bar ...
0
votes
1
answer
189
views
Shap values and Pytorch: error in plotting
I'm starting to use shap values with pytorch. For this, I just tried to run an easy example but I get an error. The code is:
import numpy as np
import torch
from torch import nn, optim
from torch.nn ...