Fix Issue <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See -- Removed the labels parameter in hammingloss function. Removed all instances of labels within the method. Since hammingloss is used only with mulitlabel input, changed len(labels) to [1]. First time contributing to open source! Please let me know what I can do to improve! <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --[MRG] Handling parameter labels removal from hammingloss <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Fix Issue Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --<!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


regression in ColumnTransformer in in 0.20.1 with columns=pd.Index This came up in one of my teaching notebooks actually (and might be in my book as well). This is very natural because columns are of type pd.Index, and so if you take some subset of columns from ` you'll now run into this error. So... 0.20.2?


traintestsplit excepting negative integers and floats The following minimal example doesn't fail: Is it a bug or indented feature? According to the docs neither of the above should make sense. See for reference for example.


Showing micro-average in classification report is confusing This is a follow up on . I don't think it makes sense to include the micro-average for multi-class classification. The three columns will always show the same value, all of which being the same as accuracy. I find that confusing. If you want to show this (I don't see why you'd want to show the same number three times), I would at least make it clear in the report that it's accuracy. IncrementalPCA fails if data size % batch size < ncomponents throws The error occurs because the last batch generated by may be smaller than . I reduced the iris data to 101 instances, so the last batch has only a single data instance, which is less than the number of components. As far as I see, none of the current unit tests run into this. ( could, if the code that raises the exception would compare with - which it should, but doesn't). Skipping the last batch if it is to small, that is, changing to fixes the problem. please confirm that this solution seems OK before I go preparing the PR and tests. No error is thrown.


Davies Bouldin measure: division by zero I'm facing a problem with the davies bouldin measure. This is the warning that I get: .local/lib/python3.7/site- RuntimeWarning: divide by zero encountered in truedivide score = (intradists[:, None] + intradists) / centroiddistances This is the implementation in sklearn: I found on stack overflow: With this implementation I don't get any warnings, but the results differ: Davies Bouldin measure: division by zero I'm facing a problem with the davies bouldin measure. This is the warning that I get: .local/lib/python3.7/site- RuntimeWarning: divide by zero encountered in truedivide score = (intradists[:, None] + intradists) / centroiddistances This is the implementation in sklearn: I found on stack overflow: With this implementation I don't get any warnings, but the results differ:


DOC: add details to QuantileTransformer documentation I think the documentation of should say how it is implemented. There is even a about it and we could take some elements of the answer. Besides I was thinking that to map to a uniform distribution, the implementation was just computing the empirical cdf of the columns but it does not seem to be the case.


fails for multioutput ensemble models with non-numeric DVs <!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Multioutput forest models assume that the dependent variables are numeric. Passing string DVs returns the following error: I'm going to take a stab at submitting a fix today, but I wanted to file an issue to document the problem in case I'm not able to finish a fix. I wrote a test based on which currently fails: No error is thrown, can run for all ensemble multioutput models <!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --I replicated this error using the current master branch of sklearn (0.21.dev0). <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


Suggestion: Add support for unpenalized logistic regression provides unpenalized OLS, and , which supports , also supports . But if you want plain old unpenalized logistic regression, you have to fake it by setting in to a large number, or use from instead.


AttributeError: 'PrettyPrinter' object has no attribute 'indentat_name' There's a failing example in , and here's a piece of code causing it: With the stack trace:


ValueError not enough values to unpack (expected 5, got 0) I'm using with in sklearn v 0.20.1. MacOS Mojave. The issue traces to these two lines below, with Setting did not work, still hitting the same issue. From the trackback, I suspect that the line below could be returning None. This doesn't always happen, but only sometimes. My param search was able to run for a few rounds, and then hit this issue. The problem could be that lightgbm did not return valid results (have not verified), but either way, this error could be handled better. Does anyone have more insight into the potential cause? Thanks.


LassoLarsIC: unintuitive copyX behaviour Hi, I would like to report what seems to be a bug in the treatment of the parameter of the class. Because it's a simple bug, it's much easier to see in the code directly than in the execution, so I am not posting steps to reproduce it. As you can see here, LassoLarsIC accepts a copyX parameter. However, it also takes a copyX parameter a few lines below, in the definition of . Now there are two values (potentially contradicting each other) for copyX and each one is used once. Therefore can have a mixed behaviour. Even worse, this can be completely invisible to the user, since copyX has a default value of True. Let's assume that I'd like it to be False, and have set it to False in the initialization, . I then call and my choice will be silently overwritten. Ideally I think that copy_X should be removed as an argument in . No other estimator seems to have a duplication in class parameters and fit arguments (I've checked more than ten in the linear models module). However, this would break existing code. Therefore I propose that takes a default value of and only overwrites the existing value if the user has explicitly passed it as an argument to . I will submit a PR to that effect.


and have inconsistent default init <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--and have inconsistent default init. has while has . See As suggested, we could change the default in with a deprecation process. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


StandardScaler fit overflows on float16 When using StandardScaler on a large float16 numpy array the mean and std calculation overflows. I can convert the array to a larger precision but when working with a larger dataset the memory saved by using float16 on smaller numbers kind of matter. The error is mostly on numpy. Adding the dtype on the mean/std calculation does it but I'm not sure if that how people here would like to do it. The normalized array


ColumnTransformer behavior for negative column indexes <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --The behavior of when negative integers are passed as column indexes is not clear. The first transformer should either raise an error or give the same result as the second transformer


MissingIndicator failed with non-numeric inputs fails with string and object type numpy arrays


Feature request: support for arbitrary bin spacing in calibration.calibrationcurve I was using , and it currently accepts an parameter to specify the number of bins to evenly partition the probability space between 0 and 1. However, I am using this in combination with a gradient boosting model in which the probabilities are very uncalibrated, and most of the predictions are close to 0. When I use the calibrated classifier, the result is very noisy because there are many data points in some bins and few, if any, in others (see example below). In the code below, I made a work-around to do what I want and show a plot of my output (in semilog space because of the skewed distribution). I haven't contributed to a large open-source project before, but if there's agreement this would be a useful feature, I would be happy to try to draft up a PR. Notice the large disparity in results between the different bins chosen. For this reason, I think the user should be able to choose the bin edges, as in numpy's or matplotlib's functions. !


sklearn.modelselection.StratifiedKFold either shuffling is wrong or documentation is misleading <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --Regarding the shuffle parameter, the documentation states: "Whether to shuffle each stratification of the data before splitting into batches". However, instead of shuffling samples within each stratum, the order of batches is shuffled. As you can see in the output below, 1 is always paired with 11, 2 with 12, 3 with 13, etc. regardless whether shuffle parameter is True or False. When shuffle=True, the batches are always the same for any randomstate, but appear in a different order. When cross-validation is performed, the results from each batch are summed and then divided by the number of batches. Changing the order of batches does not change the result. The way shuffle works now is completely useless from cross-validation perspective. import numpy as np from sklearn.modelselection import StratifiedKFold RANDOMSEED = 1 samplesperclass = 10 X = np.linspace(0, samplesperclass2-1, samplesperclass 2) y = np.concatenate(((samplesperclass), (samplesperclass)), axis=0) print(X, '\n', y, '\n') print('\nshuffle = False\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=True, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) RANDOMSEED += 1 print('\nshuffle = True, Random seed =', RANDOMSEED, '\n') kfold = StratifiedKFold(nsplits=10, shuffle=False, randomstate=RANDOMSEED) result = 0 for foldn, (trainidx, testidx) in enumerate((X, y)): print(trainidx, '\n', testidx) <!-- Example: No error is thrown. Please paste or describe the expected results.--I expect batches to be different when Shuffle is turned on for different randomstate seeds. But they are the same <!-- Please paste or specifically describe the actual output or traceback. --[ 0. 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. 11. 12. 13. 14. 15. 16. 17. 19.] [1. 1. 1. 1. 1. 1. 1. 1. 1. 1. 0. 0. 0. 0. 0. 0. 0. 0. 0. 0.] shuffle = False [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] shuffle = True, Random seed = 1 [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] shuffle = True, Random seed = 2 [ 1 2 3 4 5 6 7 8 9 11 12 13 14 15 16 17 18 19] [ 0 10] [ 0 2 3 4 5 6 7 8 9 10 12 13 14 15 16 17 18 19] [ 1 11] [ 0 1 3 4 5 6 7 8 9 10 11 13 14 15 16 17 18 19] [ 2 12] [ 0 1 2 4 5 6 7 8 9 10 11 12 14 15 16 17 18 19] [ 3 13] [ 0 1 2 3 5 6 7 8 9 10 11 12 13 15 16 17 18 19] [ 4 14] [ 0 1 2 3 4 6 7 8 9 10 11 12 13 14 16 17 18 19] [ 5 15] [ 0 1 2 3 4 5 7 8 9 10 11 12 13 14 15 17 18 19] [ 6 16] [ 0 1 2 3 4 5 6 8 9 10 11 12 13 14 15 16 18 19] [ 7 17] [ 0 1 2 3 4 5 6 7 9 10 11 12 13 14 15 16 17 19] [ 8 18] [ 0 1 2 3 4 5 6 7 8 10 11 12 13 14 15 16 17 18] [ 9 19] System: python: 3.7.2 (default, Jan 13 2019, 12:50:01) [Clang 10.0.0 (clang-1000.11.45.5)] executable: /usr/local/opt/python/bin/python3.7 machine: Darwin-18.2.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 18.1 setuptools: 40.6.3 sklearn: 0.20.2 numpy: 1.15.2 scipy: 1.2.0 Cython: None pandas: 0.23.4 <!-- Thanks for contributing! --


KBinsDiscretizer: kmeans fails due to unsorted binedges with fails in certain situations, due to centers and consequently binedges being unsorted, which is fatal for np.digitize. A very simple way to reproduce this is to set nbins in the existing testnonuniformstrategies from to a higher value (here 5 instead of 3). In this simple example it seems like an edge case to set nbins to almost the number of data points. However I've seen this happen in productive situations with very reasonable number of bins of order log2(number of unique values of X). No error is thrown. <!-- Thanks for contributing! --KBinsDiscretizer: kmeans fails due to unsorted binedges with fails in certain situations, due to centers and consequently binedges being unsorted, which is fatal for np.digitize. A very simple way to reproduce this is to set nbins in the existing testnonuniformstrategies from to a higher value (here 5 instead of 3). In this simple example it seems like an edge case to set nbins to almost the number of data points. However I've seen this happen in productive situations with very reasonable number of bins of order log2(number of unique values of X). No error is thrown. <!-- Thanks for contributing! --


GaussianMixture predict and fitpredict disagree when ninit1 When is specified in GaussianMixture, the results of fitpredict(X) and predict(X) are often different. The unit test doesn't catch this because it does not set . No exceptions.


precisionscore shows incorrect value precisionscore shows incorrect value ([[0,0,1],[0,1,0],[0,0,1]]) B=A precisionscore(A,B, average=None) array([ 0., 1., 1.]) array([ 1., 1., 1.]) array([ 0., 1., 1.]) import platform; print(platform.platform()) Darwin-14.5.0-x8664-i386-64bit import sys; print("Python", sys.version) ('Python', '2.7.10 (default, Jul 14 2015, 19:46:27) \n[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)]') import numpy; print("NumPy", numpy.version) ('NumPy', '1.13.3') import scipy; print("SciPy", scipy.version) ('SciPy', '1.0.0') import sklearn; print("Scikit-Learn", sklearn.version) ('Scikit-Learn', '0.18.1')


Different r2score multioutput default in r2score and base.RegressorMixin We've changed multioutput default in r2score to "uniformaverage" in 0.19, but in base.RegressorMixin, we still use ` (). Also see the strange things below:


Ensure monotonic bin edges for KBinsDiscretizer strategy quantile The percentiles returned from np.percentile are monotonic up to possible numeric instabilities. Monotonicity is enforced by applying a simple maximum on subsequent values to deal with this case and increase robustness. The additional line is a no-op in almost all cases. This is unfortunate, but since there is essentially no performance impact, I guess robustness is worth the effort.


gamma='scale' in SVC I believe that setting in is not meeting its intended purpose of being invariant to the scale of . Currently, is set to . However, I believe it should be . Rationale: if you scale by 10 you need to scale by 1/100, not 1/10, to achieve the same results. See the definition of the RBF kernel : the "units" of are 1/x^2, not 1/x. I also tested this empirically: scaling by 10 and scaling by 1/100 gives the same result as the original, whereas scaling by 10 and scaling by 1/10 gives a different result. Here is some code: Note that will become the default setting for in version 0.22. Related: ,


Differences among the results of KernelPCA with rbf kernel Hi there, I met with a problem: When I run KernelPCA for dimension reduction for the same datasets, the results are different in signs. Just to reduce the dimension to 7 with rbf kernel: pca = KernelPCA(ncomponents=7, kernel='rbf', copyX=False, njobs=-1) pca.fittransform(X) The same result. The results are the same except for their signs:( [[-0. -0. -0. 0. -0. -0. 0.]] [[ 0. 0. 0. -0. -0. -0. -0.]] [[-0. -0. 0. 0. 0. 0. 0.]] 18.1


ENH: support DataFrames in OneHot/OrdinalEncoder without converting to array Left-over to do from Idea is to support DataFrames without converting to a contiguous array. This conversion is not needed, as the transformer encodes the input column by column anyway, so it would be rather easy to preserve the datatypes per column. This would avoid converting a potentially mixed-dtype DataFrame (eg ints and object strings) to a full object array. This can introduces a slight change in behaviour (it can change the of the in certain edge cases, eg when you had a mixture of float and int columns). (Note that is not yet necessarily means to have special handling for certain pandas dtypes such as categorical dtype, see , in an initial step, we could still do a on each column / coerce each column to a numpy array).


partialfit does not account for unobserved target values when fitting priors to data My understanding is that priors should be fitted to the data using observed target frequencies and a variant of to avoid assigning 0 probability to targets not yet observed. It seems the implementation of does not account for unobserved targets at the time of the first training batch when computing priors. RuntimeWarning: divide by zero encountered in log self.classlogprior = ((self.classcount) - scikit-learn version: 0.20.2 This behavior is not very intuitive to me. It seems requires for the right reason, but doesn't actually offset target frequencies to account for unobserved targets.


Isolation forest - decisionfunction & averagepathlength method are memory inefficient Isolation forest consumes too much memory due to memory ineffecient implementation of anomoly score calculation. Due to this the parallelization with njobs is also impacted as anomoly score cannot be calculated in parallel for each tree. Run a simple Isolation forest with nestimators as 10 and as 50 respectively. On memory profiling, it can be seen that each building of tree is not taking much memory but in the end a lot of memory is consumed as a for loop is iteration over all trees and calculating the anomoly score of all trees together and then averaging it. - line 267-281 ` Due to this, in case of more no. of estimators(1000), the memory consumed is quite high. Possible Solution: The above for loop should only do the averaging of anomoly score from each estimator instead of calculation. The logic of isoforest anomoly score calculation can be moved to base estimator class so it is done for each tree( i guess file-similar to other method available after fitting) The memory consumption is profound as we increase no. of estimators. The fit method calls decision function & average anomoly score which are taking quite a lot memory. the memory spike is too high in the very end, that is in finall call to method. <!-- Thanks for contributing! --


checkclassweightbalancedclassifiers is never run?! checkclassweightbalancedclassifiers(name, Classifier, Xtrain, ytrain, Same for `


TypeError when supplying a boolean X to HuberRegressor fit when fitting with boolean predictors. No error is thrown when of is (second line of code in the snipped above, ) Boolean array is expected to be converted to by as it is done by, say . is thrown: Latest versions of everything as far as I am aware: <!-- Thanks for contributing! --<!-- NP! --


DOC Improve doc of nquantiles in QuantileTransformer The uses numpy.percentile(Xtrain, .) as the estimator of the quantile function of the training data. To know this function perfectly we just need to take . Then it is just a linear interpolation (which is done in the code afterwards). Therefore I don't think we should be able to choose and we should prevent users from thinking that the higher the better the transformation. As mentioned by IRL it is however true that it can be relevant to choose when is very large. I suggest to add more information on the impact of in the doc which currently reads: For example using 100 times more landmarks result in the same transformation Interestingly if you do not choose correctly, the linear interpolation done afterwards does not correspond to the numpy.percentile(Xtrain, .) estimator. This is not "wrong" as these are only estimators of the true quantile function/cdf but I think it is confusing and would be better to stick with the original estimator. For instance, the following raises an AssertionError.


returnintercept==True in ridgeregression raises an exception <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--<!-- Example: No error is thrown. Please paste or describe the expected results.--(the values can differ, but at least no exception should be raised) <!-- Please paste or specifically describe the actual output or traceback. --<!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


crossvalpredict returns bad prediction when evaluated on a dataset with very few samples returns bad prediction when evaluated on a dataset with very few samples on 1 class, causing class being ignored on some CV splits. Verified on the scikit latest dev version.


Confusing pretty print repr for nested Pipeline Taking the examples from the docs (https://scikit-) that involves some nested pipelines in columntransformer in pipeline The repr that you get for this pipeline: which I found very confusing: the outer pipeline seems to have only 1 step (the 'preprocessor', as the 'classifier' disappeared in the ). It's probably certainly not easy to get a good repr in all cases, and for sure the old behaviour was even worse (it would show the first 'imputer' step of the pipeline inside the column transformer as if it was the second step of the outer pipeline ..). But just opening this issue as a data point for possible improvements. Without knowing how the current repr is determined: ideally I would expect that, if the full repr is too long, we first try to trim it step per step of the outer pipeline, so that the structure of that outer pipeline is still visible. But that is easier to write than to code .. :) cc


labelrankingaverageprecisionscore: sampleweighting isn't applied to items with zero true labels labelrankingaverageprecisionscore offers a sampleweighting argument to allow nonuniform contribution of individual samples to the reported metric. Separately, individual samples whose labels are the same for all classes (all true or all false) are treated as a special case (precision == 1, ). However, this special case bypasses the application of sampleweight (). So, in the case where there is both non-default sampleweighting and samples with, for instance, zero labels, the reported metric is wrong. <!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--See example in Average of AP of first and second samples = 0.625 Sum of AP of all three samples, divided by sum of weighting vector = 2.25/2 = 1.125 System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /usr/bin/python3 machine: Linux-4.14.79+-x8664-with-Ubuntu-18.04-bionic BLAS: macros: SCIPYMKLH=None, HAVECBLAS=None libdirs: /usr/local/lib cblaslibs: mklrt, pthread Python deps: pip: 19.0.3 setuptools: 40.8.0 sklearn: 0.20.3 numpy: 1.14.6 scipy: 1.1.0 Cython: 0.29.6 pandas: 0.22.0 <!-- Thanks for contributing! --


Confusing error message in OrdinalEncoder when passing single list of categories Small example: The error message is still using the old , which makes it very confusing. (another issue is that we might be able to actually detect this case)


Implement RMSE (root-mean-square error) metric and scorer RMSE seems to be a popular metric but now one has to calculate it through `. Wiki page:


GradientBoostingRegressor initial estimator does not play together with Pipeline Using a pipeline as the initial estimator of GradientBoostingRegressor doesn't work due to incompatible signatures. The signature of is which cannot be called with three positional arguments as above. So I guess line 1421 in should read instead and the issue is solved. Right?


Be more tolerant in checkarray for CalibratedClassifierCV For our package , the function is very convenient for Weakly Supervised Learners, as it can make PairsClassifier estimators return a probability for a pair of points to be labeled as similar or dissimilar, when those return a decision function. However, we currently cannot use it because our inputs can be 3D (example: ), and uses with default parameters that does not allow 3D inputs. However, other meta-estimators like do not call , so we can use them easily in metric-learn. Is the in really useful or could we do without it ? If we could, I'd be happy to provide a PR to do so


Expose warmstart in Isolation forest It seems to me that supports incremental addition of new trees with the parameter of its parent class, . Even though this parameter is not exposed in , it gets inherited from and one can use it by changing it to after initialization. To make it work, you have to also increment on every iteration. It took me a while to notice that it actually works, and I had to inspect the source code of both and . Also, it looks to me that the behavior is in-line with that is behind e.g. . To make it more easier to use, I'd suggest to: expose in , default ; document it in the same way as it is documented for , i.e. say: add a test to make sure it works properly; possibly also mention in the "IsolationForest example" documentation entry;


Comparing string to array in estimatemi In ` but discrete features can be an array of indices or a boolean mask. This will error in future versions of numpy. Also this means we never test this function with discrete features != 'auto', it seems?


improve error message when passing sampleweight to Pipeline Many estimators take a parameter named . does not, since it wants its parameters to be prefixed by the step name with a delimiter: This error message is not friendly enough. It should explicitly describe the correct format for passing to a step in a Pipeline.


Stratified subsampler utility? I have some data and that I want to subsample (i.e. only keep a subset of the samples) in a stratified way. Using something like is almost what I need. But that will error if: I happen to want exactly samples () I want something close to which is not enough to have a stratified test or train set () Basically what I need is a option to , that's probably the most appropriate place to introduce this.


Numerical precision of euclideandistances with float32 <!-- Instructions For Filing a Bug: --I noticed that sklearn.metrics.pairwise.pairwisedistances function agrees with when using np.float64 arrays, but disagrees when using np.float32 arrays. See the code snippet below. I expect that the results from sklearn.metrics.pairwise.pairwisedistances would agree with for both 64-bit and 32-bit. In other words, I expect the following output: The code snippet above produces the following output for me: [WIP] Stable and fast float32 implementation of euclideandistances Superseds PR These commits implement a block-wise casting to float64 and uses the older code to compute the euclidean distance matrix on the blocks. This is done useing only a fixed amount of additional (temporary) memory. This code implements several optimizations: since the distance matrix is symmetric when , copy the blocks of the upper triangle to the lower triangle; compute the optimal block size that would use most of the allowed additional memory; cast blocks of to float64; precompute blocks of if not given so it gets reused through the iterations over ; swap and when is given, but not . Note that all the optimizations listed here have proven useful in a benchmark. The hardcoded amount of additional memory of 10MB is also derived from a benchmark. As a side bonus, this implementation should also support float16 out of the box, should scikit-learn support it at some point. Add a test for numeric precision (see ) Surprisingly bad precision, isn't it? Note that the traditional computation sqrt(sum((x-y)*2)) gets the results exact. <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --<!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Allow setting of initial hyperparameters of BayesianRidge <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--I propose to be able to set initial values of the hyperparameters (alpha, lambda) in BayesianRidge().fit. I tried to fit a sinusoidal curve with polynomials by Bayesian ridge regression, but the default initial values did not work well (left figure). So, I corrected the fit method of the BayesianRidge class so that I could set the initial value, and the regression worked well (right figure). ! <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --The code I wrote is Here. <!-- Example: No error is thrown. Please paste or describe the expected results.--The right figure <!-- Please paste or specifically describe the actual output or traceback. --The left figure <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --System: python: 3.6.6 (default, Mar 8 2019, 18:24:30) [GCC 7.3.0] executable: /home/*/.pyenv/versions/3.6.6/bin/python machine: Linux-4.15.0-47-generic-x8664-with-debian-buster-sid BLAS: macros: NOATLASINFO=1, HAVECBLAS=None libdirs: /usr/lib/x8664-linux-gnu cblaslibs: cblas Python deps: pip: 19.0.3 setuptools: 41.0.0 sklearn: 0.20.3 numpy: 1.16.2 scipy: 1.2.1 Cython: 0.29.6 pandas: 0.24.2 <!-- Thanks for contributing! --


Bug in Gradient Boosting: Feature Importances do not sum to 1 I found conditions when Feature Importance values do not add up to 1 in ensemble tree methods, like Gradient Boosting Trees or AdaBoost Trees. This error occurs once the ensemble reaches a large number of estimators. The exact conditions depend variously. For example, the error shows up sooner with a smaller amount of training samples. Or, if the depth of the tree is large. When this error appears, the predicted value seems to have converged. But it’s unclear if the error is causing the predicted value not to change with more estimators. In fact, the feature importance sum goes lower and lower with more estimators thereafter. Consequently, it's questionable if the tree ensemble code is functioning as expected. Here's sample code to reproduce this: Output: In fact, if we examine the tree at each staged prediction, we'll see that the feature importance goes to 0 after we hit a certain number of estimators. (For the code above, it's 712.) Here's code to describe what I mean: Output: I wonder if we’re hitting some floating point calculation error. BTW, I've posted this issue on the mailing list . There aren't a lot of discussion, but others seem to think there's a bug here too. Hope we can get this fixed or clarified. Thank you! -Doug Windows-7;'Python', '2.7.9 ;'NumPy', '1.9.2';'SciPy', '0.15.1';'Scikit-Learn', '0.16.1'


brierscoreloss error Hello. I think I found a mistake in brierscoreloss. When you have a target = 1 and a prediction = 1 too, brierscoreloss should be 0 (best result), but it gives 1. Why is it happening? Because checkbinaryprobabilisticpredictions gets target with only one class and convert it to 0. And metric calculates for target = 0 and prediction = 1. The same problem for target = 1 and prediction = 0. brierscoreloss is 0 (the best result), but it should be 1. Examples: Approx = [0, 0, 0, 0] Target = [1, 1, 1, 1] Weight = [1, 1, 1, 1] brierscoreloss(Target, Approx, sampleweight=Weight) result is 0 Approx = [1, 1, 1, 1] Target = [1, 1, 1, 1] Weight = [1, 1, 1, 1] brierscoreloss(Target, Approx, sampleweight=Weight) result is 1 Maybe we should fix it? Thank you.


CountVectorizer with custom analyzer ignores input argument Example: Same for . Not sure if this should be fixed or just documented; I don't like changing the behavior of the vectorizers yet again...


VarianceThreshold doesn't remove feature with zero variance When calling VarianceThreshold().fittransform() on certain inputs, it fails to remove a column that has only one unique value. The Variance threshold should produce This issue arose when I was using VarianceThreshold on a real dataset (of which this is a subset). It appears to work correctly in other situations (for instance I can't reproduce this behaviour if I replace the first column with 1's). System python: 3.5.6 (default, Aug 26 2018, 16:30:03) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE401/final)] machine: Darwin-18.2.0-x8664-i386-64bit executable: /anaconda3/envs/tensorflow/bin/python3 BLAS macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps setuptools: 40.2.0 numpy: 1.15.4 sklearn: 0.20.0 Cython: None scipy: 1.1.0 pandas: 0.24.0 pip: 19.0.1


ncomponents kwarg missing in SpectralClustering The kwarg defined in the function allow the user to choose how many eigenvalues/eigenvectors should be used in the classification. However this kwarg cannot be accessed/modified when using class, and is set to default ( = ). Since the class calls the spectralclustering function, I guess the user should be able to access this kwarg ?


Voting estimator will fail at fit if weights are passed and an estimator is None Because we don't check for an estimator to be in support, is failing`.


Handle 'drop' together with None to drop estimator in VotingClassifier/VotingRegressor As mentioned in the following , the and should accept to drop an estimator from the ensemble is the same way that is doing now.


sklearn.cluster.AffinityPropagation doesn't support sparse affinity matrix <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --sklearn.cluster.AffinityPropagation doesn't support sparse affinity matrix. <!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--A similar question is at . It focuses on default affinity. no error raised since it works for dense matrix. Traceback (most recent call last): File "<stdin", line 1, in <moduleFile "D:\Miniconda\lib\site-packages\sklearn\cluster\", line 381, in fit , verbose=self.verbose, returnniter=True) File "D:\Miniconda\lib\site-packages\sklearn\cluster\", line 115, in affinitypropagation preference = (S) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3336, in median overwriteinput=overwriteinput) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3250, in ureduce r = func(a, kwargs) File "D:\Miniconda\lib\site-packages\numpy\lib\", line 3395, in median return mean(part[indexer], axis=axis, out=out) File "D:\Miniconda\lib\site-packages\numpy\core\", line 2920, in mean out=out, kwargs) File "D:\Miniconda\lib\site-packages\numpy\core\", line 85, in mean ret = (ret / rcount) ValueError: setting an array element with a sequence. System: python: 3.6.7 (default, Oct 28 2018, 19:44:12) [MSC v.1915 64 bit (AMD64)] executable: D:\Miniconda\ machine: Windows-7-6.1.7601-SP1 BLAS: macros: SCIPYMKLH=None, HAVECBLAS=None libdirs: D:/Miniconda\Library\lib cblaslibs: mkl_rt Python deps: pip: 18.1 setuptools: 40.6.2 sklearn: 0.20.1 numpy: 1.15.4 scipy: 1.1.0 Cython: None pandas: 0.23.4 <!-- Thanks for contributing! --


AttributeError thrown when calling metrics.pairwisedistances with binary metrics and Y is None thrown when calling with binary metrics if is . No error. Should return a of shape containing the pairwise distances. This worked correctly in sklearn version 0.20.3. I think the problem was introduced in there is now a code path where has its checked without any prior check as to whether is .


pairwisedistances returns zeros for metric cosine when executed in parallel returns a list of zeros when calculating with equal to -1 or greater than 2. Using calculates the expected results. Using the metric returns non-zero results, but the values seem to be integers instead of floats. I executed with different values for and . The outputs were as follows:


Untreated overflow (?) for float32 in euclideandistances new in sklearn 21.1 I am using euclidean distances in a project and after updating, the result is wrong for just one of several datasets. When comparing it to one can see that in version 21.1 it behaves substantially different to 20.3. The matrix is an ndarray with size (100,) with float32. The data are in this zip Can be found when using sklearn 20.3, both behave identical. When using version 21.1 has many 0 entries and some unreasonably high entries Sklearn 21 System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn21/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblaslibs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.21.1 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None For sklearn 20.3 the versions are: System: python: 3.6.7 (default, Oct 22 2018, 11:32:17) [GCC 8.2.0] executable: /home/lenz/PycharmProjects/pyrolmm/venvsklearn20/bin/python3 machine: Linux-4.15.0-50-generic-x8664-with-Ubuntu-18.04-bionic BLAS: macros: HAVECBLAS=None, NOATLASINFO=-1 libdirs: /usr/lib/x8664-linux-gnu cblas_libs: cblas Python deps: pip: 9.0.1 setuptools: 39.0.1 sklearn: 0.20.3 numpy: 1.16.3 scipy: 1.3.0 Cython: None pandas: None <!-- Thanks for contributing! --


GBDTs should bin train and validation data separately? In the new GBDTs we bin the data before calling (for early-stopping). That means that the validation set is also used to find the bin thresholds (it is of course not used to find the split points!). I feel like the "data leak" is very minimal, but it seems more correct to bin Xtrain and Xval separately. WDYT?


IncrementalPCA should accept sparse input <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --is by design suited to application to sparse data in a way that most PCA classes are not. However, it is not written to accept this by default. No error should be thrown. I'd be happy to submit this as a PR if it's desirable. <details</details<!-- Thanks for contributing! --


Design of addindicator in SimpleImputer may fail when running cross validation <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--The design of depends on missing values exist in the training data. This will break cross validation. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --<!-- Example: No error is thrown. Please paste or describe the expected results.--No error is thrown. <!-- Please paste or specifically describe the actual output or traceback. --The original design was adopted because, if the training data has no missing value, there will be a column with all zeros. This type of error will appear when we try to do grid search over the parameter. One way to work around this is to split the data in such a way that missing values are available (for each column that has a missing value) in both the training set and test set. <!-- Thanks for contributing! --


Zero division error in HistGradientBoosting NaN This dataset is a bit weird in that it has 100 classes with 16 samples each. The default parameter don't work very well but we should fail more gacefully. cc


IndexError: list index out of range in exporttext when the tree only has one feature <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --returns when there is single feature. <!-- Thanks for contributing! --


ARD Regressor accuracy degrades when upgrading Scipy 1.2.1 -1.3.0 Hi, bit of a tricky one, I'm hoping someone will have some time and/or suggestions for further investigation! There seems to be an often-occurring worsening of performance (i.e. accuracy, although run-time increases too!) from the ARD regressor when upgrading from Scipy 1.2.1 -1.3.0. On a very simple dataset (see code snippets below) where a near-perfect fit should be achievable, typical error seems to degrade from order 1E-5 to 1E-2. Notably, convergence iterations seem to increase also from ~a few (~5) to around 50-200 iterations. Here's the headline plot, plotting absolute co-efficient error when fit across 1000 datasets generated with different random seeds: ! Note how with Scipy==1.2.1, errors are largely constrained to <0.01, while with Scipy==1.3.0 they range up to 0.05 (and in a few rare cases the algorithm produces garbage results, see later). I guess this could be (probably is?) a Scipy rather than Sklearn issue, but probably the only way to confirm / isolate that would be to start here. It's also possible that this worsening of behaviour is a weirdness of my particular toy example, but the difference in behaviour seems large and unexpected enough to warrant further investigation, I'd hope! OK, so here's a short snippet on just a single seed if you're curious to try this yourself. I'm generating three vectors of normally distributed values, 250 samples. Then the target is just a perfect copy of one of those vectors (index=1). We measure the accuracy of the fit by simply checking how close that coefficient is to 1.0 (the other coefficients always shrink to 0., as you'd hope): Scipy 1.2.1: Scipy 1.3.0 It could be that there's some oddity of the random data from a single seed, so I set up some short scripts to first generate a static collection of 1000 of the datasets as seen above, then collate the results from both versions of scipy. The snippets are as follows: Make data: Test sklearn: Plot results: A little investigating summary statistics of those datasets in notebook gives the following points of comparison:


IndexError thrown with LogisticRegressionCV and refit=False The following error is thrown when trying to estimate a regularization parameter via cross-validation, without refitting. No error is thrown.


NCA fails in GridSearch due to too strict parameter checks NCA checks its parameters to have a specific type, which can easily fail in a GridSearch due to how param grid is made. Here is an example: The issue is that for : 1 is not a float, and for : np.int64 is not int Before proposing a fix for this specific situation, I'd like to have your general opinion about parameter checking. I like this idea of common parameter checking tool introduced with the NCA PR. What do you think about extending it across the code-base (or at least for new or recent estimators) ? Currently parameter checking is not always done or often partially done, and is quite redundant. For instance, here is the input validation of lda: most params aren't checked and for those who are there's a lot of duplicated code. A propose to be upgrade the new tool to be able to check open/closed intervals (currently only closed) and list membership. The api would be something like that: where validoptions would be a dict of . e.g for the param of , it can be either a float or a string in a list, which would give Sometimes a parameter can only be positive or within a given interval, e.g. of must be between 0 and 1, which would give positivity of e.g. would be .


AdaBoost's "SAMME" algorithm uses 'predict' while fitting and 'predictproba' while predicting probas Subj. This seems to me to be a wrong approach, moreover this drives to such mistakes: <preAdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) NotImplementedError Traceback (most recent call last) <ipython-input-108-1d666912dadain <module() ----1 AdaBoostClassifier(algorithm="SAMME", baseestimator=SVC()).fit(trainX, trainY).predictproba(testX) /Library/Python/2.7/site- in predictproba(self, X) 716 proba = sum(estimator.predictproba(X) w 717 for estimator, w in zip(self.estimators, --718 self.estimatorweights)) 719 720 proba /= () /Library/Python/2.7/site- in <genexpr((estimator, w)) 715 else: # self.algorithm == "SAMME" 716 proba = sum(estimator.predictproba(X) w --717 for estimator, w in zip(self.estimators, 718 self.estimatorweights)) 719 /Library/Python/2.7/site- in predictproba(self, X) 493 if not self.probability: 494 raise NotImplementedError( --495 "probability estimates must be enabled to use this method") 496 497 if not in ('csvc', 'nusvc'): NotImplementedError: probability estimates must be enabled to use this method </pre


[MRG] Fix 'SparseSeries deprecated: scipy-dev failing on travis' <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: -- Issue: SparseSeries deprecated: scipy-dev failing on travis <!-- Example: See also . Please use keywords (e.g., Fixes) to create link to the issues or pull requests you resolved, so that they will automatically be closed when your pull request is merged. See --Use a Series with sparse values instead instead of . <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Add joblib in showversions joblib should be to the dependencies listed in showversions or to the issue template when sklearn version is 0.20.


Named col indexing fails with ColumnTransformer remainder on changing DataFrame column ordering I am using ColumnTransformer to prepare (impute etc.) heterogenous data. I use a DataFrame to have more control on the different (types of) columns by their name. I had some really cryptic problems when downstream transformers complained of data of the wrong type, while the ColumnTransformer should have divided them up properly. I found out that ColumnTransformer silently passes the wrong columns along as when specifying columns by name, using the option, and using DataFrames where column ordering can differ between and In this case, the wrong columns are passed on to the downstream transformers, as the example demonstrates: When using ColumnTransformer with a DataFrame and specifying columns by name, should reference the same columns when fitting and when transforming (['b'] in above example), regardless of the column positions in the data during fitting and transforming. appears to, during fitting, remember remaining named DataFrame columns by their numeric index (not by their names), which (silently) leads to the wrong columns being handled downstream if the transformed DataFrame's column ordering differs from that of the fitted DataFrame. Position in module where the index is determined: My current workaround is to not use the option but specify all columns by name explicitly. System: python: 3.7.3 (default, Mar 30 2019, 03:44:34) [Clang 9.1.0 (clang-902.0.39.2)] executable: machine: Darwin-17.7.0-x8664-i386-64bit BLAS: macros: NOATLASINFO=3, HAVECBLAS=None libdirs: cblaslibs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.3.0 Cython: None pandas: 0.24.2


plotpartialdependence() fails when used on DecisionTreeRegressor <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --fails when using a as the estimator. The problem appears to be related to the presence of a attribute (with a value of ) on the estimator, despite it being a regressor and not a classifier. Deleting the attribute from the estimator allows to successfully run. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --No error is thrown. <!-- Example: No error is thrown. Please paste or describe the expected results.--<!-- Please paste or specifically describe the actual output or traceback. --A is thrown: <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.show_versions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --<!-- Thanks for contributing! --


Pickling Tokenizers fails due to use of lambdas Cannot pickle a using the builtin python module, likely due to the use of lambdas in Example: Program runs without error Traceback: Instead of the builtin , use , which can capture the expression. <!-- Please run the following snippet and paste the output below. For scikit-learn = 0.20: import sklearn; sklearn.showversions() For scikit-learn < 0.20: import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Version information: I think this is similar to issues: (looking at the stackoverflow thread at , it suggests using which also seems to work for the toy example) Naively, I would make one of the two changes below, but I am not familiar with the scikit-learn codebase, so they might not be appropriate: the FAQ to direct people to other serialization libraries (perhaps I missed this recommendation?), e.g. at or the use of the lambdas in the vectorizer and replace them with locally def'd functions. I suspect that this solution is flawed because it doesn't account for other uses of lambdas elsewhere in the codebase, and the only complete solution would be to stop using lambdas, but these are a useful language feature.


PLS reports "array must not contain nan" if a feature is constant Originally reported at by Reproduce with: The obscure error message is due to the presence of a column containing only 0.


Need for conversion with SAG Running the following code snippet will lead to an error: I am wondering if we are not to strict here, and we should include a since that will expect a C-contiguous numpy array.


Cloning custom transform replaces values in init dictionary <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--Let us say we have a custom transform that has some arguments. When the is instantiated, these arguments are set in the init. When we clone (as happens in , for example), the arguments get copied successfully. However, if the arguments are sent to a structure such as a dictionary, the clone replaces them with None. In cases where None does not cause errors, this creates a silent error, as the cloned version of will run, producing different results from its original version (which is how I ran into this problem in the first place). Fully replicable example follows. <!-- Thanks for contributing! --


[BUG] Optics float minsamples NN instantiation None yet. When passing minsamples as a float to optics l439 & 440 execute to bring it into integer ranges, but don't convert to int: When instantiating the NearestNeighbours class with a float it raises due to the float (l448). Error message: Fix: the int(...) is for backwards compatibbility to Python 2 where with T Number, while Python3 <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Copy param ignored in TfidfVectorizer I was playing with vectorizers and I found this: However that parameter is not used later in the method. Here is used: Is there anything I am missing?


RFC ColumnTransformer input validation and requirements There have been some issues around ColumnTransformer input requirements that I think we might want to discuss more explicitly. Examples are an actual bug when changing columns: and how to define number of input features . Related is also the idea of checking for column name consistency: Mainly I'm asking whether it makes sense to have the same requirements for ColumnTransformer as for other estimators. ColumnTransformer is the only estimator that addresses columns by name, and so reordering columns doesn't impact the model and actually everything works fine. Should we still aim for checking for reordering of columns? Right now, , which makes sense, since it only cares about the columns it is using. Should we allow that going forward as long as remainder is not used? (if remainder is used, the result of would have a different shape and downstream estimators would break, so I think we shouldn't support it in this case). In either case, what's ` or the number of columns actually used? Does it even make sense to define it if we allow adding other columns that are ignored?


LassoCV always sets precompute to False before fitting the chosen alpha value I'm using a very large data-set. After fitting 100 x 3-fold cross-validated LASSOs at lightning speed (a few seconds total), LassoCV stalls at the final hurdle: fitting a LASSO with the chosen alpha value to the whole data-set (waiting over half an hour - it should only take approximately 50% longer than a single fold...). After a lot of head-scratching I found the reason why. In 's () just before calling the final () (line 1223 in Python2.7/sklearn0.19.0), there is the rather inconspicuous line So even if you've specified precompute as True when calling LassoCV, it is ignored. Why is this? It's making the computation impractically slow (should it even be this slow without precompute?) - literally just commenting the line out makes the fit instantaneous. Am I missing something here mathematically - I can't see anything wrong with using a precomputed Gram matrix for the final fit when it was used for all of the cross-validation fits? The implementation seems to imply it should be used for performance whenever numsamples num_features. Why hard set it to False?


AttributeError with crossvalpredict(method='predictproba') when using MultiOuputClassifier I believe there is a bug when using with a . I think the problem is in the use of here: To obtain the attribute of a , you need instead. If core team members have any idea of how to address this, I am happy to submit a patch. Array with prediction probabilities. System: python: 3.6.8 (default, Feb 21 2019, 18:30:04) [MSC v.1916 64 bit (AMD64)] executable: C:\Users\nak142\Miniconda3\envs\myo\ machine: Windows-10-10.0.-SP0 BLAS: macros: libdirs: cblas_libs: cblas Python deps: pip: 19.1.1 setuptools: 41.0.1 sklearn: 0.21.2 numpy: 1.16.4 scipy: 1.2.1 Cython: 0.29.12 pandas: 0.24.2


StratifiedKFold makes fold-sizes very unequal I found this when trying to write tests for . Right now, `: We could achieve something like but our rounding doesn't let us :-/


kernelapproximation.Nystroem does not support precomputed kernel The documentation says that precomputed kernels are supported in Nystroem, but in reality it does not seem to be the case: https://scikit- Example code: Leads to


HistGradientBoostingClassifier does not work with string target when early stopping turned on <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--The scorer used under the hood during early stopping is provided with being integer while are original classes (i.e. string). We need to encode each time that we want to compute the score. <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --No error is thrown <!-- Please paste or specifically describe the actual output or traceback. --Maybe one solution would be to do:


medianabsoluteerror multioutput Multioutput is not currently supported in . Is this a design choice or has it just not been implemented yet? In case of the latter, I am happy to submit a PR.


datasets :: makeclassification() weights parameter should be a sequence (not just a list). If there is a pertinent reason that must be a list, while all other iterable parameters are arrays, then it should be mentioned in the docstring. Otherwise, the docstring should be amended as in , e.g. "weights : list of floats or None (default=None)" -"weights : sequence of floats or None (default=None)", along with amended lines 165 and 171 (see Corrections). Should not fail: as valid as a list: #wimlds [MRG] Added an example to the …or class <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: --<!-- Contributes to . --I an example to the () <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


IterativeImputer behaviour on missing nan's in fit data Why is this behaviour forced: Features with missing values during transform which did not have any missing values during fit will be imputed with the initial imputation method only. This means by default it will return the mean of that feature. I would prefer just fit one iteration of the chosen estimator and use that fitted estimator to impute missing values. Actual behaviour: Example - The second feature missing --mean imputation Example adjusted - Second feature has values --iterative imputation with estimator Maybe line 679 to 683 should be optional with a parameter like force-iterimpute.


HGBC with categoricalcrossentropy fails silently on binary classification gives: And works fine. should either generalize or raise an error on binary classification. Ping


Fitting TransformedTargetRegressor with sampleweight in Pipeline Can't fit a using . May be link to ? Example: ! Fitting with <!-- Thanks for contributing! --


ZeroDivisionError in sparsefit for SVM with empty supportvectors When using sparse data, in the case where the supportvectors attribute is be empty, fitsparse gives a ZeroDivisionError No error is thrown and


Documentation section 3.3.1.1 has incorrect description of brierscoreloss In the documentation, section 3.3.1.1. "Common cases: predefined values" includes the remark As far as I can tell, this is true for all of the listed metrics, except the . In the case of , a lower loss value is better. This is because measures the mean-square difference between a predicted probability and a categorical outcome; the Brier score is minimized at 0.0 because all summands are either or when the model is making perfect predictions. On the other hand, the Brier score is maximized at 1.0 when all predictions are opposite the correct label, as all summands are either or . Therefore, the definition of the is not consistent with the quotation from section 3.3.1.1. I suggest making 2 changes to relieve this confusion. a function which simply negates the value of ; this is a direct analogy to what is done in the case of . A better model has a lower value of log-loss (categorical cross-entropy loss), therefore a larger value of the negative log-loss implies a better model. Naturally, the same is true for Brier score, where it is also the case that a better model is assigned a lower loss. reference to from section 3.3.1.1. Brier score is useful in lots of ways; however, because it does not have the property that a larger value implies a better model, it seems confusing to mention it in the context of section 3.3.1.1. References to can be replaced with , which has the property that better models have large values, just like accuracy, ROC AUC and the rest of the listed metrics.


Error could be improved with DummyClassifier constant strategy when constant value not in training data Error: We could add in the error message what constant value was provided (in this case ) and what the possible values are. This could be something like this (improvement more than welcome): Context: this was seen during the EuroScipy tutorial. The adult census dataset classes has a space in it at the beginning and the provided value did not have the space. Putting what the provided value was and what were the possible values would have helped the user fixing the problem.


RepeatedKFold and RepeatedStratifiedKFold do not show correct repr string and do not show correct \repr\ string.


data leak in GBDT due to warm start (This is about the non-histogram-based version of GBDTs) X is split into train and validation data with . As noted, in a warm starting context, this will produce a leak if If is a instance: some samples that were used for training in a previous might be used for validation now.


Deprecate classes attribute in DecisionTreeRegressor This partially relates to issue Currently, if you fit a decision tree regressor, and call the attribute , it will return none. This attribute does not appear on the doc string and shouldn't. This was surfaced from an issue related to mismatch attributes ( ) Reviewed the and worked with on different options including using a decorator on a property but it triggered the deprecation message when calling fit which was bad. In this PR, the was changed to in the parent. And a test was to the


MultiTaskLassoCV with fitintercept=True returns wrong results There is something wrong with and binary features. It always returns the same mse for all the alphas and hence chooses a huge regularization zeroing out all coefficients. The same holds for too. However, this doesn't happen with . Moreover it doesn't happen if I set , or if I generate random normal features. I am working on anaconda, windows system, with python 3.7.1 and with scikit-learn v0.21.3, numpy v1.16.2. Consider the following code: It returns On the other hand, if I generate normal features X, then things are good: which returns: Also weirdly if I set , then things are good even with binary features: which returns


MaxAbsScaler Upcasts Pandas to float64 <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --I am working with the Column transformer, and for memory issues, am trying to produce a float32 sparse matrix. Unfortunately, regardless of pandas input type, the output is always float64. I've identified one of the Pipeline scalers, MaxAbsScaler, as being the culprit. Other preprocessing functions, such as OneHotEncoder, have an optional argument. This argument does not exist in MaxAbsScaler (among others). It appears that the upcasting happens when is executed. Is it possible to specify a dtype? Or is there a commonly accepted practice to do so from the Column Transformer? Thank you! Example: Darwin-18.7.0-x8664-i386-64bit Python 3.6.7 (default, Jul 2 2019, 02:07:37) [GCC 4.2.1 Compatible Clang 4.0.1 (tags/RELEASE_401/final)] NumPy 1.17.1 SciPy 1.3.1 Scikit-Learn 0.20.3 Pandas 0.25.1


GridSearchCV saves all fitted estimator in cvresults['params'] when params are estimators I use GridSearchCV to optimize the hyperparameters of a pipeline. I set the param grid by inputing transformers or estimators at different steps of the pipeline, following the Pipeline documentation: I couldn't figure why dumping cvresults would take so much memory on disk. It happens that cvresults['params'] and all cvresults['param*'] objects contains fitted estimators, as much as there are points on my grid. This bug should happen only when njobs = 1 (which is my usecase). I don't think this is intended (else the arguments and attributes refit and best\estimator wouldn't be used). My guess is that during the grid search, those estimator's aren't cloned before use (which could be a problem if using the same grid search several times, because estimators passed in the next grid would be fitted...).


stripaccentsunicode fails to strip accents from strings that are already in NFKD form <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--The feature of and related does not work as expected when it processes strings that contain accents, if those strings are already in NFKD form. and should both be normalized to the same string, . is not changed, because does nothing if the string is already in NFKD form.


Inconsistent fit + transform and fittransform for FeatureUnion Is there a reason why the method signature accepts but neither nor do? It seems to go against the pattern that is the same as calling ? I see there's been discussion on supporting but it's not clear if this is resolved. My case is I'm trying to migrage code I wrote a while back where I used a Pipeline and each of my transformers adds columns to a dataframe, to a FeatureUnion where each transform only returns the new columns. One of my transforms takes a third data set in addition to X and y which is used as the transform. I guess as a workaround I'll make it a param of the transform rather than a fitarg.


meanshift and MeanShift don't have the same API I'm trying to make call (related to ) but has a parameter and uses the default, so I cannot preserve backward compatibility without adding to . Should I just do that?


Stacking: add an option to use the original dataset when training finalestimator I think it will be readonable to add an option to use the original dataset when training finalestimator. This seems reasonable and has proved to be useful in some Kaggle competitions. Reference: implementation from mlxtend usefeaturesinsecondary : bool (default: False) If True, the meta-classifier will be trained both on the predictions of the original classifiers and the original dataset. If False, the meta-classifier will be trained only on the predictions of the original classifiers.


imputationorder "ascending" and "descending" are inverted in IterativeImputer ImputerTriplets in fitted imputationsequence lists are appended according to imputationorder, but order is inverted


Use checksampleweight to consistently validate sampleweight We recently introduced which returns a validated array. We should use it consistently throughout the code base, instead of relying on custom and adhoc checks like or (which are now handled by ). Here's a list of the estimators/functions that could make use of it (mostly in or ): [x] CalibratedClassifierCV [x] DBSCAN [x] DummyClassifier [x] DummyRegressor [x] BaseBagging [x] BaseForest [x] BaseGradientBoosting [x] IsotonicRegression [x] KernelRidge [x] GaussianNB [x] BaseDiscreteNB [x] KernelDensity [x] BaseDecisionTree (I left-out the linearmodel module because it seems more involved there) Could be a decent sprint issue ? To know where a given class is defined, use e.g.


Return values of non converged affinity propagation clustering The affinity propagation Documentation states: "When the algorithm does not converge, it returns an empty array as clustercenterindices and -1 as label for each training sample." Example: I would expect that the clustering here (which does not converge) prints first an empty List and then [-1,-1,-1], however, I get [2] as cluster center and [0,0,0] as cluster labels. The only way I currently know if the clustering fails is if I use the verbose option, however that is very unhandy. A hacky solution is to check if maxiter == niter but it could have converged exactly 15 iterations before maxiter (although unlikely). I am not sure if this is intended behavior and the documentation is wrong? For my use-case within a bigger script, I would prefer to get back -1 values or have a property to check if it has converged, as otherwise, a user might not be aware that the clustering never converged. System: python: 3.6.7 (default, Nov 21 2018, 02:32:25) [GCC 4.8.2 (Red Hat 4.8.2-15)] executable: /home/jenniferh/Programs/anaconda3/envs/TFRDKit119/bin/python machine: Linux-4.15.0-52-generic-x8664-with-debian-stretch-sid BLAS: macros: SCIPYMKLH=None, HAVECBLAS=None libdirs: /home/jenniferh/Programs/anaconda3/envs/TFRDKit119/lib cblaslibs: mkl_rt, pthread Python deps: pip: 18.1 setuptools: 40.6.3 sklearn: 0.20.3 numpy: 1.15.4 scipy: 1.2.0 Cython: 0.29.2 pandas: 0.23.4


Nested Cross Validation for precomputed KNN A nested cross validation prediction using a knn with precomputed metric raised an error Should return the predictions made by the model as the following code produces: sklearn 0.20.2


regression in input validation of clustering metrics ValueError: could not convert string to float: 'b' while works with a warning? this worked in 0.21.1 without a warning (as I think it should) Edit by I removed the in the second code snippet.


ENH: add normalize parameter to metrics.classification.confusionmatrix Allows to get a normalized confusion matrix directly from the function call. I use frequently and find the need to always normalize the matrix manually maybe unnecessary. I am aware of the fact that other functions like already have this exact functionality implemented, so probably the lack of the parameter is intentional and I'm missing the why. But in case its not intentional you might find this contribution useful :).


LabelPropagation raises TypeError: A sparse matrix was passed LabelPropagation (and LabelSpreading) error out for sparse matrices. Sparse case should work as does the dense one. Changing in line 224 to seems to fix the problem for me (BTW: a similar check accepting sparse matrices is done in BaseLabelPropagations predictproba at line 189). This fix also heals LabelSpreading. FIX LabelPropagation handling of sparce matrices Label propagation and spreading allow to classify using sparse data according to documentation. Tests only covered the dense case. Newly coverage for sparse matrices allows to reproduce the problem in . The proposed fix in works for the extended tests. Supporting scipy's dokmatrix produces the UserWarning "Can't check dok sparse matrix for nan or inf.". So this format seems to be unsuitable? fails for sparse matrices


GPR enforce before calling In , sampling in the prior before calling via will assume that is made of a single target. However, this is not necessarily the case. Therefore, the shape of the output of before and after is different. In order to solve this inconsistency, we need to introduce a new parameter . Before calling this parameter should be explicitly set by the user. After , we can use the information of the target seen during without explicitly setting the parameter.


[MRG] Fixes sklearn.metrics.silhouette_samples for sparse matrices <!-- Thanks for contributing a pull request! Please ensure you have taken a look at the contribution guidelines: -- The changes update the reduce function used for computing the intra-cluster and inter-cluster distances. The current version is failing at, a) the pre-computed check for sparse matrices while getting the diagonal elements b) when trying to index a sparse matrix to pass weights to np.bincount function <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


Add meanaverageprecision Mean average precision (mAP) is a standard multi-class extension of average precision using OVR: (informationretrieval)#Meanaverage_precision Recently I prefer AP over AUC so I think it would be cool to add this. Maybe is interested? probably knows how to do this ;)


Preserving dtypes for DataFrame output by transformers that do not modify the input values It would be nice to optionally preserve the dtypes of the input using pandas output for transformers . Dtypes can contain information relevant for later steps of the analyses. E.g. if I include pd.categorical columns to represent ordinal data and then select features using a sklearn transformer the columns will loose their categorical dtype. This means I loose important information for later analyses steps. This is not only relevant for the categorical dtypes, but could expand to others dtypes (existing, future and custom). Furthermore, this would allow to sequentially use ColumnTransformer while preserving the dtypes (maybe related to ). Currently, this behavior is not given as one can see with this code snippet (minimal example for illustration purposes): Output (using sklearn version '1.2.dev0'): The ouput shows that both the and are converted to in the dataframe output. Maybe one could adjust the to also allow to preserve the dtypes. This would mean one changes the to add: another argument to . If not None the outputted dataframe uses to set the . The of the could be provided to by if the dtypes is set to preserve in the config. One could adjust specific transformers for which this might be relevant. Such a solution would need more work and does not seem to be inline with the simplicity that pandas output provides to the user for every transformer. is also interested in this feature.


IterativeImputer has no parameter "fillvalue" In the first imputation round of , an initial value needs to be set for the missing values. From its : I have set the initial strategy to . However, I want to define this constant myself. So, as I look at the parameters for I find : When strategy == “constant”, fillvalue is used to replace all occurrences of missingvalues. If left to the default, fillvalue will be 0 when imputing numerical data and “missingvalue” for strings or object data types. Based on this information, one would assume that also has the parameter , but it does not. The parameter needs to be to for when is set to . If this parameter is , please also allow as , for optimal compatibility with decision tree-based estimators. No response No response


BUG logloss renormalizes the predictions renormalizes internally such that it sums to 1. This way, a really bad model, the predictions of which do not sum to 1, gets a better loss then it actually has. Result: Result:


Make error message uniform when calling before While working , we found out that we are not consistent with the error type and message when calling before . From Please link the PRs you'll make with this issue so that I can update this list for all of us to know which estimators have been worked on: [x] AdditiveChi2Sampler() [x] Binarizer() [x] MaxAbsScaler() [x] MinMaxScaler() [x] Normalizer() [x] OrdinalEncoder() [x] PowerTransformer() [x] QuantileTransformer() [x] RobustScaler() [x] StackingClassifier() [x] StackingRegressor() [x] StandardScaler() [x] TfidfTransformer() [x] VotingClassifier() [x] VotingRegressor() [x] GaussianRandomProjection() [x] GenericUnivariateSelect() [x] IterativeImputer() [x] RFE() [x] RFECV() [x] SelectFdr() [x] SelectFpr() [x] SelectFromModel() [x] SelectFwe() [x] SelectKBest() [x] SelectPercentile() [x] SequentialFeatureSelector() [x] SparseRandomProjection() [x] VarianceThreshold() [x] KNNImputer() [x] SimpleImputer() [x] SplineTransformer() [x] DictVectorizer() [x] KBinsDiscretizer() [x] MissingIndicator() [x] IsotonicRegression() <details</detailsI assume that the most adequate error should be a asking to fit the estimator. WDYT?


sklearn.setconfig(transformoutput="pandas") breaks TSNE embeddings TSNE doesn't work when the I tracked down this bug in the sklearn codebase. The issue is here: What's happening is that returns a Pandas array under API, with the columns being named "pca0" and "pca1". So when is called, we get an IndexError, because you'd have to index with in this situation. Possible fix could be changing line 996 to this: which I am happy to make a PR to do unless somebody has a cleaner way. Cheers! No error is thrown, a 2-dimensional pandas array is returned


With MLPClassifer, when warmstart is True or coeffs are provided, fit doesn’t respect maxiters With MLPClassifer, when warmstart is True or coeffs are provided, fit doesn’t respect maxiters. The reason for this is, when fitting, max iteration check is equality (==) against When warmstart is true or coeffs are provided, initialize is not called; this method resets niter to 0. Based on this implementation, there is doubt as to the meaning of maxiter. Consider, if maxiter is 1 and fit terminates due to reaching maximum iterations, subsequent fittings with warmstart true will never terminate due to reaching maximum iterations. This is bug. An alternate interpretation is maxiter represents the maximum iterations per fit call. In this case, the implementation is also wrong. The later interpretation seems more reasonable. Iteration 1, loss = 0. ConvergenceWarning: Stochastic Optimizer: Maximum iterations reached and the optimization hasn't converged yet. Iteration 2, loss = 0. ConvergenceWarning: Stochastic Optimizer: Maximum iterations reached and the optimization hasn't converged yet. Iteration 3, loss = 0. ConvergenceWarning: Stochastic Optimizer: Maximum iterations reached and the optimization hasn't converged yet. Iteration 1, loss = 0. ConvergenceWarning: Stochastic Optimizer: Maximum iterations reached and the optimization hasn't converged yet. Iteration 2, loss = 0. Iteration 3, loss = 0. Windows-7-6.1.7601-SP1 Python 3.6.0 (v3.6.0:, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)] NumPy 1.12.0 SciPy 0.18.1 Scikit-Learn 0.18.1


CalibratedClassifierCV doesn't work with CalibratedClassifierCV with isotonic regression doesn't work when we previously set . The IsotonicRegression seems to return a dataframe, which is a problem for in where it tries to put the dataframe in a numpy array row . It should not crash.


ColumnTransformer with pandas output can't handle transformers with no features Hi, ColumnTransformer doesn't deal well with transformers that apply to 0 features (categoricalfeatures in the example below) when using "pandas" as output. It seems steps with 0 features are not fitted, hence don't appear in ( l.856) and hence break the input to the function (l.859). Here is some code to reproduce the error. If you remove .set_output(transform="pandas") on the line before last, all works fine. If you remove the ("categorical", ...) step, it works fine too. The step with no features should be ignored. Here is the error message:


RandomForestClassifier classweight/maxsamples interaction can lead to ungraceful and nondescriptive failure The acceptable values for are . One possible option for is . However, for values of near zero and , the model fails with an unhelpful error related to having an empty array as an indexer. This is only likely to come up in a grid search, as it seems unlikely that someone would deliberately force a subsample of zero items. However, it might be useful to have a minimum of one sample, or it might be useful to fail gracefully with a more descriptive error. EITHER: No error is thrown OR


Support nullable pandas dtypes in I would like to be able to pass the nullable pandas dtypes ("Int64", "Float64", "boolean") into sklearn's function. Because the dtypes become dtype when converted to numpy arrays we get : Repro with sklearn 1.2.1 We should get the same behavior as when , , and dtypes are used, which is no error: Our current workaround is to convert the data to numpy arrays with the corresponding dtype that works prior to passing it into . No response_


NDCG score doesn't work with binary relevance and a list of 1 element See this code example: It works correctly when the number of elements is bigger than 1: score and Computing does not make sense for single predictions. Throw an error if is a list of length 1 for NDCG and DCG. Test that this throws the appropriate error by running: <!-- Please be aware that we are a loose team of volunteers so patience is necessary; assistance handling other issues is very welcome. We value all user contributions, no matter how minor they are. If we are slow to review, either the pull request needs some benchmarking, tinkering, convincing, etc. or more likely the reviewers are simply busy. In either case, we ask for your understanding during the review process. For more information, see our FAQ on this topic: http://scikit- Thanks for contributing! --


MLPRegressor.partialfit produces an error when earlystopping is True WIth , when using , works fine, but partial fit produces the following error: I think this is related to this change: If early stopping is not supported for partial fit, it should handle this gracefully. If this is a bug - it should be fixed.


FutureWarning is not issued for deprecated class FutureWarning is not issued when using as a baseclass but its is not called in the subclass, here: As the comment suggest in your decorator, overriding ` in the class decorator indeed solves this issue. I'm happy to open a PR with the fix. Also, relatedly, I wonder whether you would be interested in using a generic deprecation package instead. Basically we have the same functionality in astropy (I feel it's actually has more features e.g. this works there out of the box, it helps with arg renames/removals, etc.), there is also a deprecated decorator in matplotlib, and also a very basic one in numpy. I feel that having one for the wider ecosystem would be beneficial instead of the current system where we all roll our own. At the numfocus summit I recall some interest from the mpl side, so I'm happy to get the ball rolling in this quoter if it's a thumb up from multiple projects. Darwin-17.7.0-x8664-i386-64bit Python 3.7.5 (default, Nov 1 2019, 02:16:38) [Clang 10.0.0 (clang-1000.11.45.5)] NumPy 1.19.0.dev0+ SciPy 1.4.1 Scikit-Learn 0.23.dev0 Quick fix for class deprecation decorator This is a quick and dirty fix for my use case, but looking at the astropy decorator I suspect there may be some corner cases that still doesn't work.


Setting minsamplessplit=1 in DecisionTreeClassifier does not raise exception If is set to 1, an exception should be raised according to the paramter's constraints: However, accepts without complaining. With scikit-survival 1.0, this raises an exception as expected: I suspect that this has to do with the Intervals of the constraints overlapping. satisfies the constraint, whereas the constraint should have precedence. No exception is raised.


FeatureUnion not working when aggregating data and pandas transform output selected I would like to use transform output and use a custom transformer in a feature union which aggregates data. When I'm using this combination I got an error. When I use default output it works fine. No error is thrown when using transform output.


FIX ignore nan values in partial dependence computation This PR implements the default behaviour suggested in that is ignoring values in both numerical and categorical features. Up to now, there is a bug since the computation of the percentile is impacted by the values for numerical features. In addition, introducing in the grid will potentially introduce a bias in the partial dependence computation depending on how missing values are handled in the model. Therefore, it is safe to remove them. To be consistent, then it is also safe to not include it as a category in the categorical features. In the future, we can think of adding an option to define the expected behaviour. I expect this PR to fail because we use instead of that does not use the same interpolation to compute the quantiles.


CalibratedClassifierCV fails on lgbm fitparams Hi, I'm trying to use CalibratedClassifierCV to calibrate the probabilities from a LGBM model. The issue is that when I try CalibratedClassifierCV with evalset, I get an error ValueError: Found input variables with inconsistent numbers of samples: [, 1] which is caused by checkconsistentlength function in The input to eval set is [Xvalid,Yvalid] where both Xvalid,Yvalid are arrays with different shape. Since this format is a requirement for LGBM eval set , I am not sure how will I make the checkconsistentlength pass in my scenario. Full code is given below: = (, 152) = (,) = (, 152) = (,)


X does not have valid feature names, but IsolationForest was fitted with feature names If you fit an using a it generates a warning This only seems to occur if you supply a non-default value (i.e. not "auto") for the parameter. This warning is unexpected as a) X does have valid feature names and b) it is being raised by the method but in general is supposed to indicate that predict has been called with ie. an ndarray but the model was fitted using a dataframe. The reason is most likely when you pass contamination != "auto" the estimator essentially calls predict on the training data in order to determine the parameters: Does not raise "X does not have valid feature names, but IsolationForest was fitted with feature names" raises "X does not have valid feature names, but IsolationForest was fitted with feature names"


Unable to pass splits to SequentialFeatureSelector This runs fine with e.g. , but according to the documentation, it should also be able to take an iterable of splits. However, passing splits from the cross validator fails Im fairly certain I have done similar things in the past to other classes in scikit-learn requiring a parameter. If somebody could confirm wether this is a bug, or I'm doing something wrong, that would great. Sorry if this is unneeded noise in the feed. Expected to run without errors


Thresholds can exceed 1 in while providing probability estimate While working on , I found out that something was odd with that returns a threshold greater than 1. A non-regression test (that could be part of ) could be as follow: The reason is due to the following: Basically, this is to add a point for and . However, the rule does not make sense in the case is a probability estimate. I am not sure what would be the best fix here. A potential workaround would be to check in which case we should clip to not be above 1.


AdaBoost: deprecation of "baseestimator" does not handle "baseestimator=None" setting properly Scikit-learn 1.2 deprecated 's in favour of (see ). Because there are also validators in place, old code that explicitly defined stopped working. A solution that fixes the deprecation is to add a possible to a list allowed values in ; I will do that in a PR. No error is thrown.


failing when featurenames supplied folks, I'm not sure why this works for but not for Can anyone help?


Incorrect documentation for behavior on BaseForest-derived classes The documentation states: This is also true for all classes that derive from , such as and . However, the does not reflect this behavior. When , it does not fit a new forest and instead just recomputes the OOB score if applicable. There are two potential fixes: the documentation to state: the actual behavior of this method to fit a new forest in the case where to reflect the existing documentation.


ignores the if it's an estimator When using on a , it sets the output to its sub-transformers but it ignores the transformer defined in . This issue causes the following to fail when gathering the results: Thus not gathering the final result correctly.


PowerTransformer fails with unhelpful stack trace with all-nan feature and method='box-cox' throws a difficult-to-debug error if x contains an all-nan column. Either no error is thrown and the all-nan column is preserved, or a descriptive error is thrown indicating that there is an unfittable column


partialdependence should respect sample weights Currently, the inspect.partialdependence funtions calculate arithmetic averages over predictions. For models fitted with sample weights, this is between suboptimal and wrong. Add new argument "sampleweight = None". If vector of right length, replace arithmetic average of predictions by weighted averages. Note that this does not affect the calculation of ICE curves, just the aggregate. No response No response_


partial AUC I suggest adding partial AUC to the metrics. this would compute the area under the curve up to a specified FPR (in the case of the ROC curve). this measure is important for comparing classifiers in cases where FPR is much more important than TPR. The partial AUC should also allow applying the McClish correction. see here: http://cran.r-


Stronger common tests for setting init params? / checkestimator In a solution was proposed that did something like The common tests let this pass, though that should wreck havoc on and . I haven't looked into it but I think the tests should fail on this.


Isomap and LocallyLinearEmbedding do not accept sparse matrix input (contrary to documentation) The mentions that should support sparse matrix. The error comes from the 5 of from . If documentation is correct should be called with  I can submit a PR. also accepts sparse matrix according to documentation on and methods. Given that also uses the arpack solver, I guess that it also should accept sparse matrices. Check of checkarray calls in the manifold subfolder For reference, my backtrace


KMeans gives slightly different result for njobs=1 vs. njobs 1 <!-- If your issue is a usage question, submit it here instead: StackOverflow with the scikit-learn tag: Mailing List: For more information, see User Questions: http://scikit- --<!-- Instructions For Filing a Bug: --<!-- Example: Joblib Error thrown when calling fit on LatentDirichletAllocation with evaluateevery 0--I noticed that gives a slightly different result depending on if or . <!-- Example: If the code is too long, feel free to put it in a public gist and link it in the issue: --Below is the code I used to run the same clustering on a varying number of jobs. <!-- Example: No error is thrown. Please paste or describe the expected results.--Should expect the the clustering result (e.g. the inertia) to be the same regardless of how many jobs are run in parallel. <!-- Please paste or specifically describe the actual output or traceback. --The case has a (slightly) different inertia than the parallel cases. <!-- Please run the following snippet and paste the output below. import platform; print(platform.platform()) import sys; print("Python", sys.version) import numpy; print("NumPy", numpy.version) import scipy; print("SciPy", scipy.version) import sklearn; print("Scikit-Learn", sklearn.version) --Darwin-16.7.0-x86_64-i386-64bit Python 3.6.1 (default, May 11 2017, 13:04:09) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] NumPy 1.13.1 SciPy 0.19.1 Scikit-Learn 0.20.dev0 <!-- Thanks for contributing! --


Bug: the predict method of Pipeline object does not use the exact predict method of final step estimator I am trying to use Pipeline with a customized final step estimator. This final estimator predict method can output std when using returnstd=True. But the predict method of Pipeline does not allow returnstd option, gives error on scikit- Line 54. In the user guide:user guide http://scikit- says the following, but the predict method in Pipeline is not the same as that in final estimator """ Calling fit on the pipeline is the same as calling fit on each estimator in turn, transform the input and pass it on to the next step. The pipeline has all the methods that the last estimator in the pipeline has, i.e. if the last estimator is a classifier, the Pipeline can be used as a classifier. If the last estimator is a transformer, again, so is the pipeline. """


sklearn.manifold.t_sne.trustworthiness should allow custom metric boolean parameter should be replaced by more standard .