[Numpy-discussion] Governance model request

Post by Travis Oliphant
After long conversations at BIDS this weekend and after reading the
entire governance document, I realized that the steering council is
very large and I don't agree with the mechanism by which it is
chosen.

Hmmm, well I never had the impression that the steering council would be
huge. But maybe you are right, and if it is, I could imagine something
like option 2, but vote based (could possibly dual use those in charge
of NumFOCUS relations, we had even discussed this possibility) which
would have final say if necessary (could mean that the contributers
definition could be broadened a bit).
However, I am not sure this is what you suggested, because for me it
should be a regular vote (if just because I am scared of having to make
the right pick). And while I will not block this if others agree, I am
currently not comfortable with either picking a BDFL (sorry guys :P) or
very fond of an oligarchy for live.

Anyway, I still don't claim to have a good grasp on these things, but
without a vote, it seems a bit what Matthew warned about.

One thing I could imagine is something like an "Advisory Board", without
(much) formal power. If we had a voted Steering Council, it could be the
former members + old time contributers which we would choose now. These
could be invited to meetings at the very least.

Just my current, probably not well thought out thoughts on the matter.
But neither of your three options feel very obvious to me unfortunately.

- Sebastian

Post by Travis Oliphant
A one year time frame is pretty short on the context of a two decades
old project and I believe the current council has too few people who
have been around the community long enough to help unstuck difficult
situations if that were necessary.
I would recommend three possible adjustments to the steering council
concept.
1 - define a BDFL for the council. I would nominate chuck Harris
2 - limit the council to 3 people. I would nominate chuck, nathaniel,
and pauli.
3 - add me as a permanent member of the steering council.
Writing NumPy was a significant amount of work. I have been working
it. While I don't actively participate all the time, I still have a
lot of knowledge, context, and experience in how NumPy is used, why it
is the way it is, and how things could be better. I also work with
people directly who have and will contribute regularly.
I am formally requesting that the steering council concept be adjusted
in one of these three ways.
Thanks,
Travis
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Sebastian Berg

2015-09-21 09:39:54 UTC

Hmmm, well I never had the impression that the steering council would be
huge. But maybe you are right, and if it is, I could imagine something
like option 2, but vote based (could possibly dual use those in charge
of NumFOCUS relations, we had even discussed this possibility) which
would have final say if necessary (could mean that the contributers
definition could be broadened a bit).
However, I am not sure this is what you suggested, because for me it
should be a regular vote (if just because I am scared of having to make
the right pick). And while I will not block this if others agree, I am
currently not comfortable with either picking a BDFL (sorry guys :P) or
very fond of an oligarchy for live.
Anyway, I still don't claim to have a good grasp on these things, but
without a vote, it seems a bit what Matthew warned about.
One thing I could imagine is something like an "Advisory Board", without
(much) formal power. If we had a voted Steering Council, it could be the
former members + old time contributers which we would choose now. These
could be invited to meetings at the very least.
Just my current, probably not well thought out thoughts on the matter.
But neither of your three options feel very obvious to me unfortunately.
- Sebastian

Though, maybe you should be in the steering council in any case even by
the current rules. Maybe you were not too active for a while, but I
doubt you will quite stop doing stuff on numpy soon....

Post by Travis Oliphant
Writing NumPy was a significant amount of work. I have been working
it. While I don't actively participate all the time, I still have a
lot of knowledge, context, and experience in how NumPy is used, why it
is the way it is, and how things could be better. I also work with
people directly who have and will contribute regularly.
I am formally requesting that the steering council concept be adjusted
in one of these three ways.
Thanks,
Travis
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Travis Oliphant

2015-09-21 16:20:33 UTC

I wrote my recommendations quickly before heading on a plane. I hope the
spirit of them was caught correctly. I also want to re-emphasize that I
completely understand that the Steering Council is not to be making
decisions that often and almost all activity will be similar to it is now
--- discussion, debate, proposals, and pull-requests --- that is a good
thing.

However, there is a need for leadership to help unstick things and move the
project forward from time to time because quite often doing *something* can
be better than trying to please everyone with a voice. My concerns about
how to do this judgment have 2 major components:

1) The need for long-term consistency --- a one-year horizon on defining
this group is too short in my mind for a decades-old project like NumPy.
2) The group that helps unstick things needs to be small (1, 3, or 5 at the
most)

We could call this group the "adjudication group" rather than the "Steering
Council" as well. I could see that having a formal method of changing
that "adjudication group" would be a good idea as well (and perhaps that
formal vote could be made by a vote of a group of active contributors. In
that case, I would define active as having a time-window of 5 years instead
of just 1).

Thanks,

-Travis

Hmmm, well I never had the impression that the steering council would be
huge. But maybe you are right, and if it is, I could imagine something
like option 2, but vote based (could possibly dual use those in charge
of NumFOCUS relations, we had even discussed this possibility) which
would have final say if necessary (could mean that the contributers
definition could be broadened a bit).
However, I am not sure this is what you suggested, because for me it
should be a regular vote (if just because I am scared of having to make
the right pick). And while I will not block this if others agree, I am
currently not comfortable with either picking a BDFL (sorry guys :P) or
very fond of an oligarchy for live.
Anyway, I still don't claim to have a good grasp on these things, but
without a vote, it seems a bit what Matthew warned about.
One thing I could imagine is something like an "Advisory Board", without
(much) formal power. If we had a voted Steering Council, it could be the
former members + old time contributers which we would choose now. These
could be invited to meetings at the very least.
Just my current, probably not well thought out thoughts on the matter.
But neither of your three options feel very obvious to me unfortunately.
- Sebastian

Though, maybe you should be in the steering council in any case even by
the current rules. Maybe you were not too active for a while, but I
doubt you will quite stop doing stuff on numpy soon....

_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Nathaniel Smith

2015-09-22 07:11:33 UTC

For reference, the rules for steering council membership were taken
directly from those used by the Jupyter project, and their steering
council currently has 10 people, making it larger than the "seed
council" proposed in the numpy document:
https://github.com/jupyter/governance/blob/master/people.md

We could call this group the "adjudication group" rather than the "Steering Council" as well. I could see that having a formal method of changing that "adjudication group" would be a good idea as well (and perhaps that formal vote could be made by a vote of a group of active contributors. In that case, I would define active as having a time-window of 5 years instead of just 1).

I may be misreading things, but I'm getting the impression that the
active "adjudication group" you envision is radically different from
the "steering council" as envisioned by the current governance
document. It also, I think, radically different from anything I've
ever seen in a functioning community-run FOSS project and frankly it's
something where if I saw a project using this model, it would make me
extremely wary about contributing.

The key point that I think differs is that you envision that this
"adjudication group" will actually intervene into discussions and make
formal decisions in situations other than true irreconcilable crises,
which in my estimation happen approximately never. The only two kinds
of F/OSS projects that I can think of that run like this are (a)
projects that are not really community driven at all, but rather run
as internal company projects that happen to have a public repository,
(b) massive projects like Debian and Fedora that have to manage
literally thousands of contributors, and thus have especially robust
backstop procedures to handle the rare truly irreconcilable situation.

E.g., the Debian CTTE acts as an "adjudication group" in the way it
sounds like you envision it: on a regular basis, irreconcilable
arguments in Debian get taken to them to decide, and they issue a
ruling. By some back of the envelope calculations, it looks like they
issue approximately ~0.002 rulings per debian-contributor-year [1][2].
If we assume crudely that irreconcilable differences scale linearly
with the size of a project, this suggests that a ~20 person project
like NumPy should require a ruling ~once every 20 years.

Or quoting myself from the last thread about this [3]:
] Or on the other end of things, you have e.g. Subversion, which had an
] elaborate defined governance system with different levels of
] "core-ness", a voting system, etc. -- and they were 6 years into the
] project before they had their first vote. (The vote was on the crucial
] technical decision of whether to write function calls like "f ()" or
] "f()".)

These are two real projects and how they really work. And even in
projects that do have a BDFL, the successful ones almost never use
this power to actually "unstick things" (i.e., use their formal power
to resolve a discussion). Consider PEP 484, Guido's somewhat
controversial type hints proposal: rather than use his power to move
the debate along, he explicitly delegated his power to one of the
idea's strongest critics [4].

Of course, things to get stuck. But the only time that getting them
unstuck needs or even benefits from the existence of a formal
"unsticking things" group is if the group is actually using some
powers that they have. In 99.9% of cases, though, the correct way to
get things unstuck is for a new idea to be introduced, or for someone
respected to act as a mediator, or for someone to summarize the
situation and isolate some core of agreement that allows for forward
progress. But all of *these* things can and should be done by anyone
and everyone who can contribute them -- restricting them to a small
"adjudication group" makes no sense.

In terms of real-world examples: From my point of view, the worst
parts of the NA fiasco was caused by the decision to cut off debate
with a "ruling". (Specifically, that instead of working on
implementing bitpattern-NAs -- which did have consensus -- then Mark
would go off and work on the masked-NA strategy, and eventually that
it should be merged, despite it not having consensus. I note that dynd
supports only bitpattern-NAs.)

OTOH, probably the most difficult technical debate we've had recently
is issue 5844, about the potential interactions between __binop__
methods and __numpy_ufunc__, and this is a debate that AFAICT
certainly would not have benefited from the existence of an
adjudication body. As it happens, all three of your proposed
adjudication-body-members actually are in the debate anyway; if they
hadn't been, then the very last thing that debate needs is someone
wading in without any understanding of the complex technical issues
and trying to force some resolution.

I'm not saying that truly irreconcilable problems never arise. But
given that they're so rare, and that realistically any system that
could solve them would need to be backed by basically the same group
of people that are in the current governance document's "steering
council" (because in an OSS project it's the people doing the work
that ultimately have the power), there's no point in building an
elaborate voting system and method for defining the electorate just to
handle these cases -- and quite a bit of harm. So the proposed system
is basically the simplest one that could work. (I described this line
of reasoning in more detail in [3].)

I think it would really help if you could provide some examples of
successful community-driven projects and/or specific debates that
benefited from having an adjudication body like you envision?

I also suspect this fundamental difference in how we view the role of
a governance body explains why I am unbothered by the idea of steering
council membership "timing out" after 2 years of inactivity. The
steering council's job is to informally manage things day to day, and
in extreme cases to make judgement calls. These two things both
crucially require a broad and up-to-date understanding of the issues
facing the project, ongoing debates, personalities, etc. But the
council is never intended to make any judgement call on its own; the
whole idea of the structure is to make sure that decisions are based
on as broad a scope of expertise as possible. In particular we
regularly get historical insight from Chuck, Ralf, Pauli, Robert Kern,
David Cournapeau, Pearu Peterson, Anne Archibald, Matthew Brett,
Stefan van der Walt...
and it doesn't matter at all whether they're on the council or not.
(If it did matter, that would be terrible, right? Why would you want
to *not* listen to any of those people?) If you plan to become more
active and give your perspective on things more then that's awesome
and welcome, but AFAICT this particular point is pretty orthogonal to
the composition of the steering council.

See also this comment from Fernando about the Jupyter steering
council, which happened by chance to cross my email this morning:
https://github.com/jupyter/governance/pull/6#issuecomment-142036050

-n

[1] https://www.debian.org/devel/tech-ctte#status
[2] https://contributors.debian.org/
[3] https://mail.scipy.org/pipermail/numpy-discussion/2015-September/073537.html
[4] See e.g. the bottom of this message from Nick Coghlan, where he
talks about how PEP "pronouncements" are done -- emphasizing in
particular (my paraphrase) that the point of a BDFL/BDFL-delegate is
not to resolve any substantive issues, and that a common strategy is
to choose the person who's most skeptical of the idea to be the
BDFL-delegate:
http://thread.gmane.org/gmane.comp.python.distutils.devel/23867

--
Nathaniel J. Smith -- http://vorpus.org

Travis Oliphant

2015-09-22 08:24:37 UTC

I actually do agree with your view of the steering council as being usually
not really being needed. You are creating a straw-man by indicating
otherwise. I don't believe a small council should do anything *except*
resolve disputes that cannot be resolved without one. Like you, I would
expect that would almost never happen --- but I would argue that
extrapolating from Debian's experience is not actually relevant here.

So, if the steering council is not really needed then why have it at all?
Let's just eliminate the concept entirely.

But there are real questions that have to have an answer or an approach to
making a decision. The answer to these questions cannot really be a vague
notion of "lack of vigorous opposition by people who read the mailing list"
which then gets parried about as "the community decided this." The NumPy
user base is far, far larger than the number of people that read this list.

For better or for worse, we will always be subject to the "tyranny of who
has time to contribute lately". Fundamentally, I would argue that this
kind of "tyranny" should at least be tempered by additional considerations
from long-time contributors who may also be acting more indirectly than is
measured by a simple git log.

So, what are the questions that have to have an answer that even calls for
some kind of governance? I know Nathaniel's document listed some of them
(and perhaps all of them). Here are mine:

1) who gets commit rights to the repo (and who has them removed)?
2) who tags the release of NumPy?
3) how is it decided where money is spent if it's donated to the project
(and not just to people directly)?
4) how is it decided if someone needs to be removed from participation in
the group because they are not adding to the conversation (we have been
fortunate that this hasn't happened in NumPy before --- but it could)?
5) how is it decided what goes on the NumPy website --- i.e. will
advertisers be able to put their logos or book-covers there?

If I understand what you are proposing, then basically the "steering
council" decides these things. Perhaps rather than a steering council,
though, we just need clear answers to questions like the above --- which
might be handled differently for different questions. I don't think
these questions have very easy answers.

Ultimately NumPy has relied on and continues to rely on the mutual respect
of all the people that have worked on the code and tried to make it better.
We all have opinions about how things have gone in the past, and what
has gone well and what hasn't. But, nothing you have said persuades me
that you have a full picture of past history with respect to a lot of the
difficult kinds of conversations that have happened and the different modes
of activity that have tried to help move NumPy along. In fact, I think
you mis-understand and mis-interpret that history quite often.

I'm convinced you are well-intentioned and doing the very best you can, and
I'm very grateful that you are passionate and eager about moving NumPy
forward. Ultimately I hope it will help things.

Here is my attempt at a proposal for how to answer the above questions:

1) who gets commit rights to the repo (and who has them removed)?

* people who contribute regularly are granted commit rights by another
committer with at least two additional nominations and the lack of a veto
within 1 week of the proposal.
* nobody has commit rights removed except by unanimous consent of all the
other committers (with consent being implied if not responded to within 2
weeks).

2) who tags the release of NumPy?

* whomever volunteers to be release manager and if there is no veto from
committers.

3) how is it decided where money is spent if it's donated to the project
(and not just to people directly)?

* three people who self-select represent NumPy to Numfocus following the
rules of Numfocus (that there is only one representative from any
organization).
* If 3 committers oppose one of those people and nominate another in
place, then that person is replaced.

4) how is it decided if someone needs to be removed from participation in
the group because they are not adding to the conversation (we have been
fortunate that this hasn't happened in NumPy before --- but it could)?

* unanimous consent of all committers (with a 2 week period given for
consent to be given --- and it is assumed given if they are not heard
from).

5) how is it decided what goes on the NumPy website --- i.e. will
advertisers be able to put their logos or book-covers there?

* only Numfocus can advertise and put their logo on the website

Now, I'm sure one can poke holes in the above --- and I would welcome
better answers to the above questions. Perhaps we should just decide how
specific decisions get made and make a document that lists that and only
talks about committers instead of inventing another "bit" to differentiate
people in the community.

-Travis

Post by Travis Oliphant
I wrote my recommendations quickly before heading on a plane. I hope

the spirit of them was caught correctly. I also want to re-emphasize
that I completely understand that the Steering Council is not to be making
decisions that often and almost all activity will be similar to it is now
--- discussion, debate, proposals, and pull-requests --- that is a good
thing.

Post by Travis Oliphant
However, there is a need for leadership to help unstick things and move

the project forward from time to time because quite often doing *something*
can be better than trying to please everyone with a voice. My concerns

Post by Travis Oliphant
1) The need for long-term consistency --- a one-year horizon on defining

this group is too short in my mind for a decades-old project like NumPy.

Post by Travis Oliphant
2) The group that helps unstick things needs to be small (1, 3, or 5 at

the most)
For reference, the rules for steering council membership were taken
directly from those used by the Jupyter project, and their steering
council currently has 10 people, making it larger than the "seed
https://github.com/jupyter/governance/blob/master/people.md

Post by Travis Oliphant
We could call this group the "adjudication group" rather than the

"Steering Council" as well. I could see that having a formal method of
changing that "adjudication group" would be a good idea as well (and
perhaps that formal vote could be made by a vote of a group of active
contributors. In that case, I would define active as having a time-window
of 5 years instead of just 1).
I may be misreading things, but I'm getting the impression that the
active "adjudication group" you envision is radically different from
the "steering council" as envisioned by the current governance
document. It also, I think, radically different from anything I've
ever seen in a functioning community-run FOSS project and frankly it's
something where if I saw a project using this model, it would make me
extremely wary about contributing.
The key point that I think differs is that you envision that this
"adjudication group" will actually intervene into discussions and make
formal decisions in situations other than true irreconcilable crises,
which in my estimation happen approximately never. The only two kinds
of F/OSS projects that I can think of that run like this are (a)
projects that are not really community driven at all, but rather run
as internal company projects that happen to have a public repository,
(b) massive projects like Debian and Fedora that have to manage
literally thousands of contributors, and thus have especially robust
backstop procedures to handle the rare truly irreconcilable situation.
E.g., the Debian CTTE acts as an "adjudication group" in the way it
sounds like you envision it: on a regular basis, irreconcilable
arguments in Debian get taken to them to decide, and they issue a
ruling. By some back of the envelope calculations, it looks like they
issue approximately ~0.002 rulings per debian-contributor-year [1][2].
If we assume crudely that irreconcilable differences scale linearly
with the size of a project, this suggests that a ~20 person project
like NumPy should require a ruling ~once every 20 years.
] Or on the other end of things, you have e.g. Subversion, which had an
] elaborate defined governance system with different levels of
] "core-ness", a voting system, etc. -- and they were 6 years into the
] project before they had their first vote. (The vote was on the crucial
] technical decision of whether to write function calls like "f ()" or
] "f()".)
These are two real projects and how they really work. And even in
projects that do have a BDFL, the successful ones almost never use
this power to actually "unstick things" (i.e., use their formal power
to resolve a discussion). Consider PEP 484, Guido's somewhat
controversial type hints proposal: rather than use his power to move
the debate along, he explicitly delegated his power to one of the
idea's strongest critics [4].
Of course, things to get stuck. But the only time that getting them
unstuck needs or even benefits from the existence of a formal
"unsticking things" group is if the group is actually using some
powers that they have. In 99.9% of cases, though, the correct way to
get things unstuck is for a new idea to be introduced, or for someone
respected to act as a mediator, or for someone to summarize the
situation and isolate some core of agreement that allows for forward
progress. But all of *these* things can and should be done by anyone
and everyone who can contribute them -- restricting them to a small
"adjudication group" makes no sense.
In terms of real-world examples: From my point of view, the worst
parts of the NA fiasco was caused by the decision to cut off debate
with a "ruling". (Specifically, that instead of working on
implementing bitpattern-NAs -- which did have consensus -- then Mark
would go off and work on the masked-NA strategy, and eventually that
it should be merged, despite it not having consensus. I note that dynd
supports only bitpattern-NAs.)
OTOH, probably the most difficult technical debate we've had recently
is issue 5844, about the potential interactions between __binop__
methods and __numpy_ufunc__, and this is a debate that AFAICT
certainly would not have benefited from the existence of an
adjudication body. As it happens, all three of your proposed
adjudication-body-members actually are in the debate anyway; if they
hadn't been, then the very last thing that debate needs is someone
wading in without any understanding of the complex technical issues
and trying to force some resolution.
I'm not saying that truly irreconcilable problems never arise. But
given that they're so rare, and that realistically any system that
could solve them would need to be backed by basically the same group
of people that are in the current governance document's "steering
council" (because in an OSS project it's the people doing the work
that ultimately have the power), there's no point in building an
elaborate voting system and method for defining the electorate just to
handle these cases -- and quite a bit of harm. So the proposed system
is basically the simplest one that could work. (I described this line
of reasoning in more detail in [3].)
I think it would really help if you could provide some examples of
successful community-driven projects and/or specific debates that
benefited from having an adjudication body like you envision?
I also suspect this fundamental difference in how we view the role of
a governance body explains why I am unbothered by the idea of steering
council membership "timing out" after 2 years of inactivity. The
steering council's job is to informally manage things day to day, and
in extreme cases to make judgement calls. These two things both
crucially require a broad and up-to-date understanding of the issues
facing the project, ongoing debates, personalities, etc. But the
council is never intended to make any judgement call on its own; the
whole idea of the structure is to make sure that decisions are based
on as broad a scope of expertise as possible. In particular we
regularly get historical insight from Chuck, Ralf, Pauli, Robert Kern,
David Cournapeau, Pearu Peterson, Anne Archibald, Matthew Brett,
Stefan van der Walt...
and it doesn't matter at all whether they're on the council or not.
(If it did matter, that would be terrible, right? Why would you want
to *not* listen to any of those people?) If you plan to become more
active and give your perspective on things more then that's awesome
and welcome, but AFAICT this particular point is pretty orthogonal to
the composition of the steering council.
See also this comment from Fernando about the Jupyter steering
https://github.com/jupyter/governance/pull/6#issuecomment-142036050
-n
[1] https://www.debian.org/devel/tech-ctte#status
[2] https://contributors.debian.org/
[3]
https://mail.scipy.org/pipermail/numpy-discussion/2015-September/073537.html
[4] See e.g. the bottom of this message from Nick Coghlan, where he
talks about how PEP "pronouncements" are done -- emphasizing in
particular (my paraphrase) that the point of a BDFL/BDFL-delegate is
not to resolve any substantive issues, and that a common strategy is
to choose the person who's most skeptical of the idea to be the
http://thread.gmane.org/gmane.comp.python.distutils.devel/23867
--
Nathaniel J. Smith -- http://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Nathaniel Smith

2015-09-22 09:33:00 UTC

Post by Travis Oliphant
I actually do agree with your view of the steering council as being
usually not really being needed. You are creating a straw-man by
indicating otherwise. I don't believe a small council should do anything
*except* resolve disputes that cannot be resolved without one. Like you, I
would expect that would almost never happen --- but I would argue that
extrapolating from Debian's experience is not actually relevant here.

To be clear, Debian was only one example -- what I'm extrapolating from is
every community-driven F/OSS project that I'm aware of.

It's entirely possible my data set is incomplete -- if you have some other
examples that you think would be better to extrapolate from, then I'd be
genuinely glad to hear them. You may have noticed that I'm a bit of an
enthusiast on this topic :-).
So, if the steering council is not really needed then why have it at all?

Post by Travis Oliphant
Let's just eliminate the concept entirely.

In my view, the reasons for having such a council are:
1) The framework is useful even if you never use it, because it means
people can run "what if" scenarios in their mind and make decisions on that
basis. In the US legal system, only a vanishingly small fraction of cases
go to the Supreme Court -- but the rules governing the Supreme Court have a
huge effect on all cases, because people can reason about what would happen
*if* they tried to appeal to the Supreme Court.
2) It provides a formal structure for interfacing with the outside world.
E.g., one can't do anything with money or corporate contributions without
having some kind of written-down and enforceable rules for making decisions
(even if in practice you always stick to the "everyone is equal and we
govern by consensus" part of the rules).
3) There are rare but important cases where discussions have to be had in
private. The main one is "personnel decisions" like inviting people to join
the council; another example Fernando has mentioned to me is that when they
need to coordinate a press release between the project and a funding body,
the steering council reviews the press release before it goes public.

That's pretty much it, IMO.

The framework we all worked out at the dev meeting in Austin seems to
handle these cases well AFAICT.

Post by Travis Oliphant
But there are real questions that have to have an answer or an approach to
making a decision. The answer to these questions cannot really be a vague
notion of "lack of vigorous opposition by people who read the mailing list"
which then gets parried about as "the community decided this." The NumPy
user base is far, far larger than the number of people that read this list.

According to the dev meeting rules, no particularly "vigorous opposition"
is required -- anyone who notices that something bad is happening can write
a single email and stop an idea dead in its tracks, with only the steering
council able to overrule. We expect this will rarely if ever happen,
because the threat will be enough to keep everyone honest and listening,
but about the only way we could possibly be *more* democratic is if we
started phoning up random users at home to ask their opinion.

This is actually explicitly designed to prevent the situation where whoever
talks the loudest and longest wins, and to put those with more and less
time available on an equal footing.

Post by Travis Oliphant
For better or for worse, we will always be subject to the "tyranny of who
has time to contribute lately". Fundamentally, I would argue that this
kind of "tyranny" should at least be tempered by additional considerations
from long-time contributors who may also be acting more indirectly than is
measured by a simple git log.

I guess I am missing something fundamental here. Who are these long-time
contributors who will sit on your council of 1/3/5 but who don't even read
the mailing list? How will they know when their special insight is
necessary?

Post by Travis Oliphant
So, what are the questions that have to have an answer that even calls for
some kind of governance? I know Nathaniel's document listed some of them
1) who gets commit rights to the repo (and who has them removed)?
2) who tags the release of NumPy?
3) how is it decided where money is spent if it's donated to the project
(and not just to people directly)?
4) how is it decided if someone needs to be removed from participation in
the group because they are not adding to the conversation (we have been
fortunate that this hasn't happened in NumPy before --- but it could)?
5) how is it decided what goes on the NumPy website --- i.e. will
advertisers be able to put their logos or book-covers there?
If I understand what you are proposing, then basically the "steering
council" decides these things.

No, absolutely not. The proposal is that these issues are decided by open
discussion on the mailing list. (With the possible exception of #4 -- I
suspect that given an extreme situation like this, then once all other
efforts to mitigate the situation had failed the steering council would
probably feel compelled to talk things over to double-check they had
consensus before acting, and likely some part of this would have to be done
in private.)

This is pretty explicit in the document (and again, this is text and ideas
stolen directly from Jupyter/IPython):

"During the everyday project activities, council members participate in all
discussions, code review and other project activities as peers with all
other Contributors and the Community. In these everyday activities, Council
Members do not have any special power or privilege through their membership
on the Council. [...] the Council may, if necessary [do pretty much
anything, but] the Council's primary responsibility is to facilitate the
ordinary community-based decision making procedure described above. If we
ever have to step in and formally override the community for the health of
the Project, then we will do so, but we will consider reaching this point
to indicate a failure in our leadership."

If this is unclear then suggestions for improvement would certainly be
welcome.

This is pretty much how we do things now, and it has worked pretty well so
far -- people do get commit bits, releases get tagged, and everyone seems
happy with this part. I'd actually like to see a more explicit set of rules
around e.g. who gets commit bits, just to set expectations and provide a
clearer onramp for new contributors, but that's something that we can
easily discuss and make a decision on later.

All of these questions are ones that are easy enough to solve given some
framework for governance, but they do all require substantive discussion.
We're not going to do them justice if we try to solve them off-the-cuff in
this thread, and trying would just derail the underlying discussion about
how we make decisions. So I think we should stay focused on that.

Post by Travis Oliphant
Perhaps rather than a steering council, though, we just need clear
answers to questions like the above --- which might be handled differently
for different questions. I don't think these questions have very easy
answers.
Ultimately NumPy has relied on and continues to rely on the mutual respect
of all the people that have worked on the code and tried to make it better.
We all have opinions about how things have gone in the past, and what
has gone well and what hasn't. But, nothing you have said persuades me
that you have a full picture of past history with respect to a lot of the
difficult kinds of conversations that have happened and the different modes
of activity that have tried to help move NumPy along. In fact, I think
you mis-understand and mis-interpret that history quite often.

I'm honestly somewhat unclear on why having a "full picture of past
history" is necessary to contribute effectively (surely if that were the
case, then only Jim Hugunin could comment?), but if there are things I'm
missing then I'd certainly like to know. You've made comments along these
lines several times now, but unfortunately I can't do much to improve my
understanding without more concrete examples.

I'm convinced you are well-intentioned and doing the very best you can, and

Post by Travis Oliphant
I'm very grateful that you are passionate and eager about moving NumPy
forward. Ultimately I hope it will help things.

Thank you. I am also convinced that you're well-intentioned and doing the
very best you can, and grateful that you are also passionate and eager
about moving NumPy forward.

-n

--
Nathaniel J. Smith -- http://vorpus.org

Travis Oliphant

2015-09-22 10:08:09 UTC

Post by Nathaniel Smith

To be clear, Debian was only one example -- what I'm extrapolating from is
every community-driven F/OSS project that I'm aware of.
It's entirely possible my data set is incomplete -- if you have some other
examples that you think would be better to extrapolate from, then I'd be
genuinely glad to hear them. You may have noticed that I'm a bit of an
enthusiast on this topic :-).

Yes, you are much better at that than I am. I'm not even sure where I
would look for this kind of data.

Post by Nathaniel Smith
So, if the steering council is not really needed then why have it at all?

Post by Travis Oliphant
Let's just eliminate the concept entirely.

1) The framework is useful even if you never use it, because it means
people can run "what if" scenarios in their mind and make decisions on that
basis. In the US legal system, only a vanishingly small fraction of cases
go to the Supreme Court -- but the rules governing the Supreme Court have a
huge effect on all cases, because people can reason about what would happen
*if* they tried to appeal to the Supreme Court.

O.K. That is a good point. I can see the value in that.

Post by Nathaniel Smith
2) It provides a formal structure for interfacing with the outside world.
E.g., one can't do anything with money or corporate contributions without
having some kind of written-down and enforceable rules for making decisions
(even if in practice you always stick to the "everyone is equal and we
govern by consensus" part of the rules).

O.K.

Post by Nathaniel Smith
3) There are rare but important cases where discussions have to be had in
private. The main one is "personnel decisions" like inviting people to join
the council; another example Fernando has mentioned to me is that when they
need to coordinate a press release between the project and a funding body,
the steering council reviews the press release before it goes public.

O.K.

Post by Nathaniel Smith
That's pretty much it, IMO.
The framework we all worked out at the dev meeting in Austin seems to
handle these cases well AFAICT.

How did we "all" work it out when not everyone was there? This is where I
get lost. You talk about community decision making and yet any actual
decision is always a subset of the community. I suppose you just rely on
the "if nobody complains than it's o.k." rule? That really only works if
the project is moving slowly.

Post by Nathaniel Smith
But there are real questions that have to have an answer or an approach to

Post by Travis Oliphant
making a decision. The answer to these questions cannot really be a vague
notion of "lack of vigorous opposition by people who read the mailing list"
which then gets parried about as "the community decided this." The NumPy
user base is far, far larger than the number of people that read this list.

O.K. so how long is the time allowed for this kind of opposition to be
noted?

Post by Nathaniel Smith
This is actually explicitly designed to prevent the situation where
whoever talks the loudest and longest wins, and to put those with more and
less time available on an equal footing.

The long-time contributors wouldn't necessarily sit on that council. But,
I would support the idea of an advisory council that could act if it saw
things going the wrong direction. This is where those people would sit.

In the case of a 1 / 3 / 5 member council -- I would not argue to be on it
at all (but I would argue that some care should be taken to be sure it has
people with some years of experience if they are available). I'm only
asking to be on a steering council that is larger than 5 people, and I
don't actually prefer that the steering council be larger than 5 people.

Post by Nathaniel Smith
No, absolutely not. The proposal is that these issues are decided by open
discussion on the mailing list. (With the possible exception of #4 -- I
suspect that given an extreme situation like this, then once all other
efforts to mitigate the situation had failed the steering council would
probably feel compelled to talk things over to double-check they had
consensus before acting, and likely some part of this would have to be done
in private.)

O.K. Then, I am misunderstanding.

Post by Nathaniel Smith
This is pretty explicit in the document (and again, this is text and ideas
"During the everyday project activities, council members participate in
all discussions, code review and other project activities as peers with all
other Contributors and the Community. In these everyday activities, Council
Members do not have any special power or privilege through their membership
on the Council. [...] the Council may, if necessary [do pretty much
anything, but] the Council's primary responsibility is to facilitate the
ordinary community-based decision making procedure described above. If we
ever have to step in and formally override the community for the health of
the Project, then we will do so, but we will consider reaching this point
to indicate a failure in our leadership."

Granting commit rights to the repo does not seem to me to be an "everyday
project activity" --- so I suppose I was confused. I suppose that could
happen with no serious objections on the list. It seems that the people
who actually have the commit bit presently should be consulted more than
the general public, though.

Post by Nathaniel Smith
This is pretty much how we do things now, and it has worked pretty well so
far -- people do get commit bits, releases get tagged, and everyone seems
happy with this part. I'd actually like to see a more explicit set of rules
around e.g. who gets commit bits, just to set expectations and provide a
clearer onramp for new contributors, but that's something that we can
easily discuss and make a decision on later.

O.K. I think that the commit bit has not been that clear --- and was
closer when I was more active to the commiters rule I gave before.

Post by Nathaniel Smith
All of these questions are ones that are easy enough to solve given some
framework for governance, but they do all require substantive discussion.
We're not going to do them justice if we try to solve them off-the-cuff in
this thread, and trying would just derail the underlying discussion about
how we make decisions. So I think we should stay focused on that.

Post by Travis Oliphant
Perhaps rather than a steering council, though, we just need clear
answers to questions like the above --- which might be handled differently
for different questions. I don't think these questions have very easy
answers.
Ultimately NumPy has relied on and continues to rely on the mutual
respect of all the people that have worked on the code and tried to make it
better. We all have opinions about how things have gone in the past,
and what has gone well and what hasn't. But, nothing you have said
persuades me that you have a full picture of past history with respect to a
lot of the difficult kinds of conversations that have happened and the
different modes of activity that have tried to help move NumPy along. In
fact, I think you mis-understand and mis-interpret that history quite
often.

The reason is that you in particular make very long arguments that try to
persuade on the basis of your understanding of how things were done in the
past (both here and in other projects). If you don't understand the
history and the arguments that have been had before, you are throwing away
information. It's not a show-stopper, for sure, but it does potentially
make it less likely that a better solution will be found.

I will have to respond to your incorrect characterizations in another
thread.

-Travis

Chris Barker

2015-09-21 21:22:47 UTC

Post by Travis Oliphant
After long conversations at BIDS this weekend and after reading the entire
governance document, I realized that the steering council is very large

How large are we talking? I think there were 8 people named -- and I'm not
sure all 8 want to do it. But 5 or seems like a fine number -- I wonder if
defining the number is a good idea.

Post by Travis Oliphant
A one year time frame is pretty short on the context of a two decades old
project

I see:

"""
who have produced contributions that are substantial in
quality and quantity, and sustained over at least one year
"""

that could be longer, but it looks like it DOESN'T mean "have contributed
within the last year" -- so we could certainly, at this point, have a lot
of folks that have been around a long time that could qualify. A certain
guy named Travis come to mind...

and I believe the current council has too few people who have been around

Post by Travis Oliphant
the community long enough to help unstuck difficult situations if that were
necessary.

That's actually a bad sign right there -- the suggested group are the folks
that have actually been contributing lately -- too bad that apparently
isn't anyone that has been around a long time :-(

also see:
"""
This will
include but is not limited to code, code review, infrastructure work,
mailing list and chat participation, community help/building,
education and outreach, design work, etc.
"""

so it's not only people doing the actual coding ( A really small number
:-( )

But I note that the the suggestion to "seed the council" with "everyone
who has reviewed/merged a pull request since Jan 1, 2014" - I'm not sure
that gives us as diverse a council as we'd like -- but it is only the
seed...

Post by Travis Oliphant
1 - define a BDFL for the council.

Well, that would make a number of things easier -- but I think there was a
consensus that there simply was no obvious candidate -- and if the
candidate is not obvious, then they can't really be a BDFL.

Post by Travis Oliphant
2 - limit the council to 3 people. I would nominate chuck, nathaniel, and
pauli.

Good group -- but maybe 3 a bit too small.

Post by Travis Oliphant
3 - add me as a permanent member of the steering council.

I'd love to see you on the council, and there is no question of your
qualifications -- but I don't think anyone should be anymore permanent than
any other. According to the docs so far, the only thing anyone needs to do
to remain on the council is stay active -- and not piss everyone off so
much as to get a consensus from everyone else that you need to be kicked
off.

And the end of this -- I think the big question still at hand is how big
the council should be.

My experience with consensus suggests that it not be very big :-)

-Chris
--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

***@noaa.gov

Matthew Brett

2015-09-22 01:24:16 UTC

Hi Travis, and all,

You might have seen I was advocating for having someone who takes
final responsibility for the project, partly to get discussions
unstuck, as you said.

I agree with Chris, that at this stage, there is no-one who could be
Benevolent Dictator for the project. It seems to me that (nearly)
everyone would agree that, if there is a leader, we would not want a
person who dictates, but someone who is good at putting in the hard
work and time it takes to hear all the arguments, and guide people to
agreement, where that is possible. If there is a leader, I think we
should select them for those skills.

For the proposal that you join the steering committee, I see two problems.

The first is, that the members of the steering committee have to be
the people who are keeping in touch with the day to day work of the
project. I am sure you would agree that, in the past, you have not
had enough time for that. Your recent emails about the new work you
want to do, also imply that you may be too busy to get involved in the
detailed discussions needed for getting the code merged. In any case,
I think you'd also agree that in the past you have hoped that you
would have more time for numpy than you did.

The second problem is that you have a potential conflict of interest,
in that it is possible for the needs of Continuum to conflict with the
needs of numpy. I believe, from previous emails on this list, that
you don't think that is very important, but I continue to disagree
about that. For example, see this interview with Linus Torvalds,
where he talks about going out of his way to make sure that people can
trust his decisions, despite the fact he is paid to work on Linux [1].

In practice, the most obvious step that I can think of, is to defer
the decision as to whether you are on the steering committee for six
months. I guess over that time you will be working with the other
numpy developers more closely.

I should say that Stefan and I are working on a governance proposal,
in this case for scikit-image, where we split the governance into 1)
the developers doing the work and making the day to day decisions and
2) the trustees, usually from other relevant projects, or
no-longer-active developers, who make sure the project does not go off
the rails. I think you'd be an excellent and obvious trustee, in
that model.

Cheers,

Matthew

[1] http://www.bbc.com/news/technology-18419231

Bryan Van de Ven

2015-09-22 01:33:31 UTC

Post by Matthew Brett
The second problem is that you have a potential conflict of interest,
in that it is possible for the needs of Continuum to conflict with the
needs of numpy. I believe, from previous emails on this list, that
you don't think that is very important, but I continue to disagree
about that. For example, see this interview with Linus Torvalds,

Can you actually spell out some particular, articulable, concrete points of concern? Otherwise this just seems like idle, non-constructive speculation (at best).

Bryan

David Cournapeau

2015-09-22 01:42:04 UTC

Can you actually spell out some particular, articulable, concrete points
of concern? Otherwise this just seems like idle, non-constructive
speculation (at best).

There is ample history of such things happening in OSS history, so I think
that's a fair concern, even if that has not happened for numpy yet.

David

Bryan Van de Ven

2015-09-22 05:15:55 UTC

Can you actually spell out some particular, articulable, concrete points of concern? Otherwise this just seems like idle, non-constructive speculation (at best).
There is ample history of such things happening in OSS history, so I think that's a fair concern, even if that has not happened for numpy yet.

Specific examples to support that claim would be appreciated. In particular, examples where an OSS project was corrupted (is that the word?) by a company specifically at the hand of the project's original creator would be especially relevant.

Beyond that, what (even in a broad sense) is an example of a goal that "Continuum might need" that would conceivably do detriment to the NumPy community? That it be faster? Simpler to maintain? Easier to extend? Integrate better with more OS projects? Attract new active developers? Receive more financial support? Grow its user base even more?

And then, even if there is some sliver of daylight to be uncovered between Travis and the community, what is the current organizational mechanism by which a problematic contribution is forced into NumPy against the will of all other core developers?

Finally, is there any previous instance whatsoever that can be pointed to of Travis forcing some change into NumPy contrary to the interest of the community? If not, what actual basis is there to exclude him from a steering committee?

I obviously have a point of view; but my opinion here is my own, which is: this sort of pre-emptive impugning of someone's integrity is speculation upon speculation upon speculation

Bryan

Stefan van der Walt

2015-09-22 06:07:24 UTC

Post by Bryan Van de Ven
Beyond that, what (even in a broad sense) is an example of a goal that
"Continuum might need" that would conceivably do detriment to the
NumPy community? That it be faster? Simpler to maintain? Easier to
extend? Integrate better with more OS projects? Attract new active
developers? Receive more financial support? Grow its user base even
more?

I don't know how productive it is to dream up examples, but it's not
very hard to do. Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core. But it's been stated by some that,
with so many people running Conda, breaking the ABI is of little
consequence. And then it wouldn't be much of a leap to think that numba
is an acceptable dependency.

There's a broad range of Continuum projects that intersect with what
NumPy does: numba, DyND, dask and Odo to name a few. Integrating them
into NumPy may make a lot more sense for someone from Continuum than for
other members of the community.

Stéfan

Bryan Van de Ven

2015-09-22 06:28:48 UTC

Post by Stefan van der Walt
I don't know how productive it is to dream up examples, but it's not

Well, agreed, to be honest.

Post by Stefan van der Walt
very hard to do. Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core. But it's been stated by some that,

Who are you speaking for? The entire community? Under what mandate?

Post by Stefan van der Walt
with so many people running Conda, breaking the ABI is of little
consequence. And then it wouldn't be much of a leap to think that numba
is an acceptable dependency.

The current somewhat concrete proposal I am aware of involves funding cleaning up dtypes. Is there another concrete, credible proposal to make Numba a dependency of NumPy that you can refer to? If not, why are we mired in hypotheticals?

Post by Stefan van der Walt
There's a broad range of Continuum projects that intersect with what
NumPy does: numba, DyND, dask and Odo to name a few. Integrating them
into NumPy may make a lot more sense for someone from Continuum than for
other members of the community.

May? Can you elaborate? More speculation. My own position is that these projects want to integrate with NumPy, not the converse. Regardless of my opinion, can you actually make any specific arguements, one way or the otehr? What if if some integrations actually make more sense for the community? Is this simply a dogmatic ideological position that anything whatsoever that benefits both NumPy and Continuum simultaneously is bad, on principle? That's fine, as such, but let's make that position explicit if that's all it is.

Bryan

Stefan van der Walt

2015-09-22 06:57:49 UTC

Hi Brian

Post by Stefan van der Walt
very hard to do. Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core. But it's been stated by some that,

Who are you speaking for? The entire community? Under what mandate?

I am speaking on behalf of myself, under no mandate.

Post by Bryan Van de Ven
The current somewhat concrete proposal I am aware of involves funding
cleaning up dtypes. Is there another concrete, credible proposal to
make Numba a dependency of NumPy that you can refer to? If not, why
are we mired in hypotheticals?

I'm sorry if I misunderstood, but I thought you wanted us to explore
hypothetical confictual situations.

Post by Bryan Van de Ven
May? Can you elaborate? More speculation. My own position is that
these projects want to integrate with NumPy, not the
converse. Regardless of my opinion, can you actually make any specific
arguements, one way or the otehr? What if if some integrations
actually make more sense for the community? Is this simply a dogmatic
ideological position that anything whatsoever that benefits both NumPy
and Continuum simultaneously is bad, on principle? That's fine, as
such, but let's make that position explicit if that's all it is.

No, I don't have such a dogmatic ideological position. I think,
however, that it is somewhat unimaginative to propose that there are no
potential conflicts whatsoever.

I am happy if we can find solutions that benefit both numpy and any
company out there. But in the end, I'm sure you'd agree that we want
the decisions that lead to such solutions to be taken in the best
interest of the project, and not be weighed by alterior motivations of
any sorts. In the end, even the *perception* that that is not the case
can be very harmful.

Stéfan

Travis Oliphant

2015-09-22 08:50:41 UTC

No, I don't have such a dogmatic ideological position. I think,
however, that it is somewhat unimaginative to propose that there are no
potential conflicts whatsoever.
I am happy if we can find solutions that benefit both numpy and any
company out there. But in the end, I'm sure you'd agree that we want
the decisions that lead to such solutions to be taken in the best
interest of the project, and not be weighed by alterior motivations of
any sorts. In the end, even the *perception* that that is not the case
can be very harmful.

I will only comment on the last point. I completely agree that the
*perception* that this is not the case can be harmful.

But, what concerns me is where this perception comes from --- from actual
evidence of anything that is not in the best interests of the project ---
or just ideological differences of opinion about the way the world works
and the perceptions around open source and markets. It is quite easy for
someone to spread FUD about companies that contribute to open source ---
and it has the effect of discouraging companies from continuing to
contribute to community projects. This removes a huge amount of
potential support from projects.

In NumPy's case in particular, this kind of attitude basically guarantees
that I won't be able to contribute effectively and potentially even people
I fund to contribute might not be accepted --- not because we can't
faithfully participate in the same spirit that we have always contributed
to SciPy and NumPy and other open source projects --- but because people
are basically going to question things just because.

What exactly do you need me to say to get you to believe that I have
nothing but the best interests of array computing in Python at heart?

The only thing that is different between me today and me 18 years ago is
that 1) I have more resources now, 2) I have more knowledge about computer
science and software architecture and 3) I have more experience with how
NumPy gets used. All I can do is continue to try and make things better
the best way I know how.

-Travis

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Nathaniel Smith

2015-09-22 07:33:12 UTC

Hi Bryan,

I understand where you're coming from, but I'd appreciate it if we
could keep the discussion on a less visceral level? Nobody's personal
integrity is being impugned, but it's the nature of this kind of
governance discussion that we have to consider unlikely-and-unpleasant
hypotheticals. It's like when you talk to a lawyer about a contract or
a will or whatever: they'll make you think about all kinds of horrible
possibilities, not because any of them are likely, but because sooner
or later *something* will go wrong, and the point of having a
contract/will/governance document is to provide some framework to
handle whichever unlikely edge case does arise.

And the other purpose of this kind of framework is to avoid the
*perception* (whether justified or not) of these kinds of conflicts of
interest -- if not handled well then this can easily scare away
volunteers, contributions from other companies, etc. Obviously you
know Travis and Continuum well as an employee there, but to most
people without that personal connection, Continuum is just a large
corporate entity with unknown motives. Imagine if instead of Continuum
we were talking about it was Google or RandomAggressiveStartup -- some
company that you didn't have any personal connection or insight into.
For someone in this position, it's not unreasonable to want more of a
reassurance that things will work out then just being asked to trust
that the CEO is personally committed to not being evil and they can
trust him.

Also, in these messages you seem to be working from a framework where
people working in good faith will always agree, and so any suggestion
of a possible conflict of interest can only arise through bad faith.
But this isn't true. Is it really so difficult to imagine that, say,
Continuum and Enthought might at some point have conflicting needs for
numpy, or for Continuum's vision of the future of numpy could be
less-than-perfectly-representative with every bit of numpy's entire
giant userbase? Continuum is a company that has in the past submitted
rather obscure patches to numpy that AFAICT are used exclusively by a
particular contracting customer (e.g. [1]), and that is currently
investing a substantial multiple of numpy's development budget on
building a direct competitor to numpy.

To emphasize: I personally am not concerned by these facts -- we did
merge that patch I linked, and there's no animosity between the numpy
and dynd teams -- but reasonable people could reasonably be concerned
that tricky situations might emerge in the future, and I've talked to
plenty of people who are nervous about Continuum's influence in
general. And with my numpy developer hat on, I don't even care which
"side" is right, that's irrelevant to me, because my concern is with
providing a space where both "sides" feel comfortable working
together. This is why it's crucial that numpy-the-project be clearly
distinguishable as an independent entity that is beholden only to its
own community: it's *exactly because* we *value* the contribution of
companies like Continuum, and want to be able to freely foster those
relationships without creating suspicion and bad blood.

Also to emphasize: none of this means that Travis can't be on the
steering council -- I think that's a more complex issue that I'll
address separately. All I'm saying is that pretending that you aren't
going to reassure people by pretending this elephant isn't in the
room, or by taking a reasonable set of concerns and aggressively
turning them into a referendum on individual people's integrity.

Finally, can I point out... anyone who has some wariness around the
possible influence of financial interests on the community (whether
justified or not!) is in particular not going to be reassured if you
keep aggressively attempting to shut down any perceived criticism of
*your own employer*. I know that your paycheck is not dictating your
opinions, and probably the hypothetical people I'm talking about are
being totally unfair to you for even considering such a thing, but...
strictly as a matter of practical rhetoric, I don't think this is the
most effective way to accomplish your goals.

-n

[1] https://github.com/numpy/numpy/pull/359

Post by Stefan van der Walt
I don't know how productive it is to dream up examples, but it's not

Well, agreed, to be honest.

Post by Stefan van der Walt
very hard to do. Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core. But it's been stated by some that,

Who are you speaking for? The entire community? Under what mandate?

Post by Stefan van der Walt
with so many people running Conda, breaking the ABI is of little
consequence. And then it wouldn't be much of a leap to think that numba
is an acceptable dependency.

--
Nathaniel J. Smith -- http://vorpus.org

Travis Oliphant

2015-09-22 10:44:12 UTC

Post by Nathaniel Smith
Hi Bryan,
I understand where you're coming from, but I'd appreciate it if we
could keep the discussion on a less visceral level? Nobody's personal
integrity is being impugned, but it's the nature of this kind of
governance discussion that we have to consider unlikely-and-unpleasant
hypotheticals. It's like when you talk to a lawyer about a contract or
a will or whatever: they'll make you think about all kinds of horrible
possibilities, not because any of them are likely, but because sooner
or later *something* will go wrong, and the point of having a
contract/will/governance document is to provide some framework to
handle whichever unlikely edge case does arise.
And the other purpose of this kind of framework is to avoid the
*perception* (whether justified or not) of these kinds of conflicts of
interest -- if not handled well then this can easily scare away
volunteers, contributions from other companies, etc. Obviously you
know Travis and Continuum well as an employee there, but to most
people without that personal connection, Continuum is just a large
corporate entity with unknown motives. Imagine if instead of Continuum
we were talking about it was Google or RandomAggressiveStartup -- some
company that you didn't have any personal connection or insight into.
For someone in this position, it's not unreasonable to want more of a
reassurance that things will work out then just being asked to trust
that the CEO is personally committed to not being evil and they can
trust him.

Anybody who comes to NumPy should know that I wrote it and then gave it to
the community -- creating an independent foundation at the same time as I
created a Company to create a division between them so that my actions
could be understood. I really don't know how to give more reassurance.

I'm actually offended that so many at BIDS seem eager to crucify my
intentions when I've done nothing but give away my time, my energy, my
resources, and my sleep to NumPy for many, many years. I guess if your
intent is to drive me away, then you are succeeding.

Of course not, but this is no different than anyone else and a company
should not be singled out. All you are doing is forcing any contribution
to be either only from a non-profit or have individuals hide their actual
allegiances.

Post by Nathaniel Smith
Continuum is a company that has in the past submitted
rather obscure patches to numpy that AFAICT are used exclusively by a
particular contracting customer (e.g. [1]), and that is currently
investing a substantial multiple of numpy's development budget on
building a direct competitor to numpy.

Good grief! These comments are so much bunk that I have to call you on it
emphatically. You claim below that you are unconcerned but yet you
insinuate some crazy, ulterior motivations for Continuum actually helping
people upgrade their NumPy installation that broke their old code because
of changes to NumPy. This kind of comment really upsets me. You
dismiss real things and real problems that happen and brush it away because
it's *commercial*.

That patch you reference was actually an attempt to fix a problem that the
community pushed on the world --- therefore breaking people's code (but
good thing the ABI didn't change...). We were up front about it and
worked with the community to get a change into NumPy to accommodate a user
of the tool. It was hard work to figure out how to do that. To have you
use that as some kind of argument against Continuum is not only unfair, it
is exactly the kind of mis-characterization and mis-interpretation of
events that I refer to in other posts.

And to say that we are investing a substantial multiple of Numpy's
development budget in a competitor is also incorrect. Continuum invests
in competent people who want to do interesting things. We don't have a
rule that says things are "off-limits" including NumPy. If competent
people feel like an alternative to NumPy is a good idea, then a certain
amount of exploration in that direction is allowed. DyND does not have
to be a competitor to NumPy. It might compete with *your* vision of
NumPy, but it doesn't have to compete with NumPy.

Post by Nathaniel Smith
To emphasize: I personally am not concerned by these facts -- we did
merge that patch I linked, and there's no animosity between the numpy
and dynd teams -- but reasonable people could reasonably be concerned
that tricky situations might emerge in the future, and I've talked to
plenty of people who are nervous about Continuum's influence in
general.

Who are these people? How about they come forward and express what it is
they are actually nervous about. Really? nervous? What kind of "tricky"
situations are we talking about. Can't you see that this sounds as odd to
me as me telling you that I'm concerned about BIDS influence? What about
Dato, Databricks, Enthought, or Cloudera influence? What does this even
mean? Is this just Matthew and Stefan or are there others as well with
these feelings? These are the only actual people who have expressed in
public what might be considered concern that I am aware of. I think this
kind of anti-commercial commentary has no place in a community that also
includes people that work at companies.

I can't see how we can agree to *any* governance document with this kind of
FUD being spread around.

Post by Nathaniel Smith
And with my numpy developer hat on, I don't even care which
"side" is right, that's irrelevant to me, because my concern is with
providing a space where both "sides" feel comfortable working
together. This is why it's crucial that numpy-the-project be clearly
distinguishable as an independent entity that is beholden only to its
own community: it's *exactly because* we *value* the contribution of
companies like Continuum, and want to be able to freely foster those
relationships without creating suspicion and bad blood.

Of course we agree on this. I have no idea why anyone thinks we don't?
That's the one thing we *do* agree on. That NumPy is an independent
project which can be influenced by anyone in the community and should be
developed based on technical discussions and not fear of hob-goblins and
people that also work at companies that may benefit from the work that goes
on here (there is a large list in this camp). I am deeply saddened by
the insinuation and the implication of what these threads are telling me
about how little my efforts have been valued by people I care about.

Post by Nathaniel Smith
Also to emphasize: none of this means that Travis can't be on the
steering council -- I think that's a more complex issue that I'll
address separately. All I'm saying is that pretending that you aren't
going to reassure people by pretending this elephant isn't in the
room, or by taking a reasonable set of concerns and aggressively
turning them into a referendum on individual people's integrity.

We should call out the elephant in the room. But, I think we should
understand who and what the elephant is. Perhaps there are too many
off-list and back-channel conversations happening at BIDS and elsewhere
that are serving to bias people against facts. Facts that otherwise would
show that I and Continuum have always just been trying to ensure the
success of NumPy as an independent project that is fully supported,
backward compatible, maintained, available to the world in easy to install
ways, and documented.

Post by Nathaniel Smith
Finally, can I point out... anyone who has some wariness around the
possible influence of financial interests on the community (whether
justified or not!) is in particular not going to be reassured if you
keep aggressively attempting to shut down any perceived criticism of
*your own employer*. I know that your paycheck is not dictating your
opinions, and probably the hypothetical people I'm talking about are
being totally unfair to you for even considering such a thing, but...
strictly as a matter of practical rhetoric, I don't think this is the
most effective way to accomplish your goals.

I agree with this. I certainly did not encourage Bryan. He was acting out
of his own sense of injustice.

But, I would add, that your insinuation and mis-characterization of my
activities at Continuum and potentially elsewhere are unfair and incorrect
and also not effective at getting governance documents approved and
ratified. I will get over feeling offended and work to get over my
frustration at academics for thrusting this anti-company and potentially
anti-Travis rhetoric on the community.

But, if you indeed have such hard feelings, then please air all of them so
we can hopefully just get past this.

Post by Nathaniel Smith
-n
[1] https://github.com/numpy/numpy/pull/359

Post by Stefan van der Walt
I don't know how productive it is to dream up examples, but it's not

Well, agreed, to be honest.

Post by Stefan van der Walt
very hard to do. Currently, e.g., the community is not ready to adopt
numba as part of the ufunc core. But it's been stated by some that,

Who are you speaking for? The entire community? Under what mandate?

Post by Stefan van der Walt
with so many people running Conda, breaking the ABI is of little
consequence. And then it wouldn't be much of a leap to think that numba
is an acceptable dependency.

The current somewhat concrete proposal I am aware of involves funding

cleaning up dtypes. Is there another concrete, credible proposal to make
Numba a dependency of NumPy that you can refer to? If not, why are we mired
in hypotheticals?

May? Can you elaborate? More speculation. My own position is that these

projects want to integrate with NumPy, not the converse. Regardless of my
opinion, can you actually make any specific arguements, one way or the
otehr? What if if some integrations actually make more sense for the
community? Is this simply a dogmatic ideological position that anything
whatsoever that benefits both NumPy and Continuum simultaneously is bad, on
principle? That's fine, as such, but let's make that position explicit if
that's all it is.

Post by Bryan Van de Ven
Bryan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
Nathaniel J. Smith -- http://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Sebastian Berg

2015-09-22 11:41:55 UTC

Post by Nathaniel Smith
Hi Bryan,
I understand where you're coming from, but I'd appreciate it if we
could keep the discussion on a less visceral level? Nobody's personal
integrity is being impugned, but it's the nature of this kind of
governance discussion that we have to consider
unlikely-and-unpleasant
hypotheticals. It's like when you talk to a lawyer about a contract or
a will or whatever: they'll make you think about all kinds of horrible
possibilities, not because any of them are likely, but because sooner
or later *something* will go wrong, and the point of having a
contract/will/governance document is to provide some framework to
handle whichever unlikely edge case does arise.
And the other purpose of this kind of framework is to avoid the
*perception* (whether justified or not) of these kinds of conflicts of
interest -- if not handled well then this can easily scare away
volunteers, contributions from other companies, etc. Obviously you
know Travis and Continuum well as an employee there, but to most
people without that personal connection, Continuum is just a large
corporate entity with unknown motives. Imagine if instead of Continuum
we were talking about it was Google or RandomAggressiveStartup -- some
company that you didn't have any personal connection or insight into.
For someone in this position, it's not unreasonable to want more of a
reassurance that things will work out then just being asked to trust
that the CEO is personally committed to not being evil and they can
trust him.
Anybody who comes to NumPy should know that I wrote it and then gave
it to the community -- creating an independent foundation at the same
time as I created a Company to create a division between them so that
my actions could be understood. I really don't know how to give more
reassurance.
I'm actually offended that so many at BIDS seem eager to crucify my
intentions when I've done nothing but give away my time, my energy, my
resources, and my sleep to NumPy for many, many years. I guess if
your intent is to drive me away, then you are succeeding.

Frankly, I am bit surprised at how this is developing. Why?! Nobody,
except being silly, really would doubt your intentions. That said, I
will be honest with you. I do not feel I know you well enough (or the
mode you operate) to for example accept you as BDFL of numpy [1] and I
frankly do not think that -- just like anyone else -- you should have
any special place in the governance document (which obviously does not
mean you should be blocked from being on the steering council) [2].

I just think there should probably not be any specific name in the NumPy
governance document at all.

The thing is, NOBODY really seems suggests that [3]. I have dislike
giving any special "power" to ANYONE. Frankly, I think if some people
are nervous (not so much those active in the discussion), it is probably
because they perceive that you have some direct power over numpy. As you
have said, this is just not true. But *because* of the lack of a
governance document, I would not be surprised if it *appears* to many
like you could wield a lot of power if you so wished. Simply because few
people really know how things are decided currently. And I think this
wrong perception is what makes some nervous.
If you say "maybe we should stop worrying about ABI" it may sound like
"two years from now we will definitely break ABI", the curse being that
it does not matter that you and those who know numpy's well know that it
is just you stressing strongly that we should seriously discuss it. I
have to admit, you sometimes sound a bit too definite in your
suggestions given your former position ;).

I hope I have not been rude here,

Sebastian

[1] I am sure you were *exactly* the right person to start numpy and be
the de-facto BDFL then, but today this is not on the table anyway.

[2] Also, lets be honest, you probably do have quite a bit of soft
influence, just by knowing the community, being at the conferences,
having NumFOCUS close by, etc.

[3] I guess you have in some sense, but I now understand that as a
suggestion to approach a different problem. And we can find another
solution for that problem!

Post by Nathaniel Smith
Also, in these messages you seem to be working from a
framework where
people working in good faith will always agree, and so any suggestion
of a possible conflict of interest can only arise through bad faith.
But this isn't true. Is it really so difficult to imagine that, say,
Continuum and Enthought might at some point have conflicting needs for
numpy, or for Continuum's vision of the future of numpy could be
less-than-perfectly-representative with every bit of numpy's entire
giant userbase?
Of course not, but this is no different than anyone else and a company
should not be singled out. All you are doing is forcing any
contribution to be either only from a non-profit or have individuals
hide their actual allegiances.
Continuum is a company that has in the past submitted
rather obscure patches to numpy that AFAICT are used
exclusively by a
particular contracting customer (e.g. [1]), and that is currently
investing a substantial multiple of numpy's development budget on
building a direct competitor to numpy.
Good grief! These comments are so much bunk that I have to call you
on it emphatically. You claim below that you are unconcerned but yet
you insinuate some crazy, ulterior motivations for Continuum actually
helping people upgrade their NumPy installation that broke their old
code because of changes to NumPy. This kind of comment really
upsets me. You dismiss real things and real problems that happen and
brush it away because it's *commercial*.
That patch you reference was actually an attempt to fix a problem that
the community pushed on the world --- therefore breaking people's code
(but good thing the ABI didn't change...). We were up front about it
and worked with the community to get a change into NumPy to
accommodate a user of the tool. It was hard work to figure out how to
do that. To have you use that as some kind of argument against
Continuum is not only unfair, it is exactly the kind of
mis-characterization and mis-interpretation of events that I refer to
in other posts.
And to say that we are investing a substantial multiple of Numpy's
development budget in a competitor is also incorrect. Continuum
invests in competent people who want to do interesting things. We
don't have a rule that says things are "off-limits" including NumPy.
If competent people feel like an alternative to NumPy is a good idea,
then a certain amount of exploration in that direction is allowed.
DyND does not have to be a competitor to NumPy. It might compete
with *your* vision of NumPy, but it doesn't have to compete with
NumPy.
To emphasize: I personally am not concerned by these facts -- we did
merge that patch I linked, and there's no animosity between the numpy
and dynd teams -- but reasonable people could reasonably be concerned
that tricky situations might emerge in the future, and I've talked to
plenty of people who are nervous about Continuum's influence in
general.
Who are these people? How about they come forward and express what
it is they are actually nervous about. Really? nervous? What kind
of "tricky" situations are we talking about. Can't you see that this
sounds as odd to me as me telling you that I'm concerned about BIDS
influence? What about Dato, Databricks, Enthought, or Cloudera
influence? What does this even mean? Is this just Matthew and
Stefan or are there others as well with these feelings? These are
the only actual people who have expressed in public what might be
considered concern that I am aware of. I think this kind of
anti-commercial commentary has no place in a community that also
includes people that work at companies.
I can't see how we can agree to *any* governance document with this
kind of FUD being spread around.
And with my numpy developer hat on, I don't even care which
"side" is right, that's irrelevant to me, because my concern is with
providing a space where both "sides" feel comfortable working
together. This is why it's crucial that numpy-the-project be clearly
distinguishable as an independent entity that is beholden only to its
own community: it's *exactly because* we *value* the
contribution of
companies like Continuum, and want to be able to freely foster those
relationships without creating suspicion and bad blood.
Of course we agree on this. I have no idea why anyone thinks we
don't? That's the one thing we *do* agree on. That NumPy is an
independent project which can be influenced by anyone in the community
and should be developed based on technical discussions and not fear of
hob-goblins and people that also work at companies that may benefit
from the work that goes on here (there is a large list in this camp).
I am deeply saddened by the insinuation and the implication of what
these threads are telling me about how little my efforts have been
valued by people I care about.
Also to emphasize: none of this means that Travis can't be on the
steering council -- I think that's a more complex issue that I'll
address separately. All I'm saying is that pretending that you aren't
going to reassure people by pretending this elephant isn't in the
room, or by taking a reasonable set of concerns and
aggressively
turning them into a referendum on individual people's
integrity.
We should call out the elephant in the room. But, I think we should
understand who and what the elephant is. Perhaps there are too many
off-list and back-channel conversations happening at BIDS and
elsewhere that are serving to bias people against facts. Facts that
otherwise would show that I and Continuum have always just been trying
to ensure the success of NumPy as an independent project that is fully
supported, backward compatible, maintained, available to the world in
easy to install ways, and documented.
Finally, can I point out... anyone who has some wariness around the
possible influence of financial interests on the community (whether
justified or not!) is in particular not going to be reassured if you
keep aggressively attempting to shut down any perceived criticism of
*your own employer*. I know that your paycheck is not
dictating your
opinions, and probably the hypothetical people I'm talking about are
being totally unfair to you for even considering such a thing, but...
strictly as a matter of practical rhetoric, I don't think this is the
most effective way to accomplish your goals.
I agree with this. I certainly did not encourage Bryan. He was
acting out of his own sense of injustice.
But, I would add, that your insinuation and mis-characterization of my
activities at Continuum and potentially elsewhere are unfair and
incorrect and also not effective at getting governance documents
approved and ratified. I will get over feeling offended and work to
get over my frustration at academics for thrusting this anti-company
and potentially anti-Travis rhetoric on the community.
But, if you indeed have such hard feelings, then please air all of
them so we can hopefully just get past this.
-n
[1] https://github.com/numpy/numpy/pull/359
On Mon, Sep 21, 2015 at 11:28 PM, Bryan Van de Ven

Post by Stefan van der Walt
I don't know how productive it is to dream up examples, but

it's not

Post by Bryan Van de Ven
Well, agreed, to be honest.

Post by Stefan van der Walt
very hard to do. Currently, e.g., the community is not

ready to adopt

Post by Stefan van der Walt
numba as part of the ufunc core. But it's been stated by

some that,

Post by Bryan Van de Ven
Who are you speaking for? The entire community? Under what

mandate?

Post by Stefan van der Walt
with so many people running Conda, breaking the ABI is of

little

Post by Stefan van der Walt
consequence. And then it wouldn't be much of a leap to

think that numba

Post by Stefan van der Walt
is an acceptable dependency.

The current somewhat concrete proposal I am aware of

involves funding cleaning up dtypes. Is there another
concrete, credible proposal to make Numba a dependency of
NumPy that you can refer to? If not, why are we mired in
hypotheticals?

Post by Stefan van der Walt
There's a broad range of Continuum projects that intersect

with what

Post by Stefan van der Walt
NumPy does: numba, DyND, dask and Odo to name a few.

Integrating them

Post by Stefan van der Walt
into NumPy may make a lot more sense for someone from

Continuum than for

Post by Stefan van der Walt
other members of the community.

May? Can you elaborate? More speculation. My own position is

that these projects want to integrate with NumPy, not the
converse. Regardless of my opinion, can you actually make any
specific arguements, one way or the otehr? What if if some
integrations actually make more sense for the community? Is
this simply a dogmatic ideological position that anything
whatsoever that benefits both NumPy and Continuum
simultaneously is bad, on principle? That's fine, as such, but
let's make that position explicit if that's all it is.

Post by Bryan Van de Ven
Bryan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
Nathaniel J. Smith -- http://vorpus.org
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion
--
Travis Oliphant
Co-founder and CEO
@teoliphant
512-222-5440
http://www.continuum.io
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Stefan van der Walt

2015-09-22 18:20:50 UTC

Hi Travis

Post by Travis Oliphant
I'm actually offended that so many at BIDS seem eager to crucify my
intentions when I've done nothing but give away my time, my energy, my
resources, and my sleep to NumPy for many, many years. I guess if your
intent is to drive me away, then you are succeeding.

I guess we've gone off the rails pretty far at this point, so let me at
least take a step back, and make sure that you know that:

- I have never doubted that your intensions for NumPy are anything but
good (I know they are!),
- I *want* the community to be a welcoming place for companies to
contribute (otherwise, I guess I'd not be such a fervent supporter of
the scientific eco-system using the BSD license), and
- I love your enthusiasm for the project. After all, that is a big part
of what inspired me to become involved in the first place.

My goal is not to spread uncertainty, fear nor doubt—if that was the
perception left, I apologize.

I'll re-iterate that I wanted to highlight a concern about the
interactions of a (somewhat weakly cohesive) community and strong,
driven personalities such as yourself backed by a formidable amount of
development power. No matter how good your intensions are, there are
risks involved in this kind of interaction, and if we fail to even
*admit* that, we are in trouble.

Lest the above be read in a negative light again, let me state it
up-front: *I don't think you will hijack the project, use it for your
own gain, or attempt to do anything you don't believe to be in the best
interest of NumPy.* What I'm saying is that we absolutely need to move
forward in a way that brings everyone along, and makes everyone rest
assured that their voice will be heard.

Also, please know that I have not discussed these matters with Nathaniel
behind the scenes, other than an informal hour-long discussion about his
original governance proposal. There is no BIDS conspiracy or attempts
at crucifixion. After all, you were an invited guest speaker at an
event I organized this weekend, since I value your opinion and insights.

Either way, let me again apologize if my suggested lack of insight hurt
people's feelings. I can only hope that, in educating me, we all learn
a few lessons.

Stéfan

Benjamin Root

2015-09-22 18:48:14 UTC

To expand on Ryan's point a bit about recusal... this is why we have a
general policy against self-merging and why peer review is so valuable. A
ban on self-merging is much like recusal, and I think it is a fantastic
policy.

As for a BDFL, I used to like that idea having seen it work well for Linux
and Python, but I have found it at odds with the policy of recusal and no
self-merging. That said, as I am sure Thomas Caswell can attest, a
non-self-merging policy can result in a lot of ideas getting stalled,
waiting for review that may or may not happen. I don't know what the
solution is, but I am sympathetic to those who are apprehensive about a
BDFL -- regardless of who is in that role.

Ben Root

Post by Matthew Brett
Hi Travis

I guess we've gone off the rails pretty far at this point, so let me at
- I have never doubted that your intensions for NumPy are anything but
good (I know they are!),
- I *want* the community to be a welcoming place for companies to
contribute (otherwise, I guess I'd not be such a fervent supporter of
the scientific eco-system using the BSD license), and
- I love your enthusiasm for the project. After all, that is a big part
of what inspired me to become involved in the first place.
My goal is not to spread uncertainty, fear nor doubtâif that was the
perception left, I apologize.
I'll re-iterate that I wanted to highlight a concern about the
interactions of a (somewhat weakly cohesive) community and strong,
driven personalities such as yourself backed by a formidable amount of
development power. No matter how good your intensions are, there are
risks involved in this kind of interaction, and if we fail to even
*admit* that, we are in trouble.
Lest the above be read in a negative light again, let me state it
up-front: *I don't think you will hijack the project, use it for your
own gain, or attempt to do anything you don't believe to be in the best
interest of NumPy.* What I'm saying is that we absolutely need to move
forward in a way that brings everyone along, and makes everyone rest
assured that their voice will be heard.
Also, please know that I have not discussed these matters with Nathaniel
behind the scenes, other than an informal hour-long discussion about his
original governance proposal. There is no BIDS conspiracy or attempts
at crucifixion. After all, you were an invited guest speaker at an
event I organized this weekend, since I value your opinion and insights.
Either way, let me again apologize if my suggested lack of insight hurt
people's feelings. I can only hope that, in educating me, we all learn
a few lessons.
StÃ©fan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Travis Oliphant

2015-09-22 23:13:57 UTC

I completely agree. I don't think self-merging is a good idea. I had
gotten used to self merging to SciPy and NumPy until roughly 2008 or 2009
when I was finally broken of that habit after stepping on a few people's
toes and learning better habits.

Post by Benjamin Root
To expand on Ryan's point a bit about recusal... this is why we have a
general policy against self-merging and why peer review is so valuable. A
ban on self-merging is much like recusal, and I think it is a fantastic
policy.
As for a BDFL, I used to like that idea having seen it work well for Linux
and Python, but I have found it at odds with the policy of recusal and no
self-merging. That said, as I am sure Thomas Caswell can attest, a
non-self-merging policy can result in a lot of ideas getting stalled,
waiting for review that may or may not happen. I don't know what the
solution is, but I am sympathetic to those who are apprehensive about a
BDFL -- regardless of who is in that role.
Ben Root

Post by Matthew Brett
Hi Travis

your

Post by Travis Oliphant
intent is to drive me away, then you are succeeding.

_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Matthew Brett

2015-09-22 18:48:57 UTC

Hi,

On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt

Post by Matthew Brett
Hi Travis

I guess we've gone off the rails pretty far at this point, so let me at
- I have never doubted that your intensions for NumPy are anything but
good (I know they are!),
- I *want* the community to be a welcoming place for companies to
contribute (otherwise, I guess I'd not be such a fervent supporter of
the scientific eco-system using the BSD license), and
- I love your enthusiasm for the project. After all, that is a big part
of what inspired me to become involved in the first place.
My goal is not to spread uncertainty, fear nor doubt—if that was the
perception left, I apologize.
I'll re-iterate that I wanted to highlight a concern about the
interactions of a (somewhat weakly cohesive) community and strong,
driven personalities such as yourself backed by a formidable amount of
development power. No matter how good your intensions are, there are
risks involved in this kind of interaction, and if we fail to even
*admit* that, we are in trouble.
Lest the above be read in a negative light again, let me state it
up-front: *I don't think you will hijack the project, use it for your
own gain, or attempt to do anything you don't believe to be in the best
interest of NumPy.* What I'm saying is that we absolutely need to move
forward in a way that brings everyone along, and makes everyone rest
assured that their voice will be heard.
Also, please know that I have not discussed these matters with Nathaniel
behind the scenes, other than an informal hour-long discussion about his
original governance proposal. There is no BIDS conspiracy or attempts
at crucifixion. After all, you were an invited guest speaker at an
event I organized this weekend, since I value your opinion and insights.
Either way, let me again apologize if my suggested lack of insight hurt
people's feelings. I can only hope that, in educating me, we all learn
a few lessons.

I'm also in favor of taking a step back.

The point is, that a sensible organization and a sensible leader has
to take the possibility of conflict of interest into account. They
also have to consider the perception of a conflict of interest.

It is the opposite of sensible, to respond to this with 'how dare you"
or by asserting that this could never happen or by saying that we
shouldn't talk about that in case people get frightened. I point you
again to Linus' interview [1]. He is not upset that he has been
insulted by the implication of conflict of interest, he soberly
accepts that this will always be an issue, with companies in
particular, and goes out of his way to address that in an explicit and
reasonable way.

Cheers,

Matthew

[1] http://www.bbc.com/news/technology-18419231

Travis Oliphant

2015-09-22 23:16:39 UTC

I am not upset nor was I ever upset about discussing the possibility of
conflict of interest. Of course it can be discussed --- but it should be
discussed directly about specific things --- and as others have said it is
generally easily handled when it actually could arise. The key is to
understand affiliations. We should not do things in the community that
actually encourage people to hide their affiliations for fear of backlash
or bias.

I was annoyed at the insinuation that conflict of interest is a
company-only problem that academics are somehow immune to.

I was upset about accusations and mis-interpretations of my activities and
those of my colleagues in behalf of the community.

Post by Matthew Brett
Hi,
On Tue, Sep 22, 2015 at 11:20 AM, Stefan van der Walt

Post by Matthew Brett
Hi Travis

your

Post by Travis Oliphant
intent is to drive me away, then you are succeeding.

I'm also in favor of taking a step back.
The point is, that a sensible organization and a sensible leader has
to take the possibility of conflict of interest into account. They
also have to consider the perception of a conflict of interest.
It is the opposite of sensible, to respond to this with 'how dare you"
or by asserting that this could never happen or by saying that we
shouldn't talk about that in case people get frightened. I point you
again to Linus' interview [1]. He is not upset that he has been
insulted by the implication of conflict of interest, he soberly
accepts that this will always be an issue, with companies in
particular, and goes out of his way to address that in an explicit and
reasonable way.
Cheers,
Matthew
[1] http://www.bbc.com/news/technology-18419231
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Bryan Van de Ven

2015-09-23 02:55:52 UTC

Post by Matthew Brett
The point is, that a sensible organization and a sensible leader has
to take the possibility of conflict of interest into account. They
also have to consider the perception of a conflict of interest.

Of course, and the policies to deal with conflicts have deal with the possibility that *anyone* at all might have conflict. But that was not your suggestion. Your suggestion was to make one individual be subject to additional scrutiny that no one else would be subject to. Please explain why should one person be singled out for a special "six-month waiting period" when the exact same possibility for conflict exists with anyone who is ever on the committee?

Post by Matthew Brett
It is the opposite of sensible, to respond to this with 'how dare you"
or by asserting that this could never happen or by saying that we
shouldn't talk about that in case people get frightened. I point you

Red herring. The objection (as many people have now voiced) is the double standard you proposed.

Post by Matthew Brett
again to Linus' interview [1]. He is not upset that he has been
insulted by the implication of conflict of interest, he soberly
accepts that this will always be an issue, with companies in
particular, and goes out of his way to address that in an explicit and
reasonable way.

Your selective quotation is impressive. Also in that interview, Linus points out that his employment contract is probably "unique in the entire world". Which means in practical terms that the details of what he has does are fairly well irrelevant to any other situation. So what is the point in bringing it up, at all, except to try and diminish someone else by comparison?

(I'm done.)

Bryan

Charles R Harris

2015-09-23 05:01:44 UTC

Of course, and the policies to deal with conflicts have deal with the
possibility that *anyone* at all might have conflict. But that was not your
suggestion. Your suggestion was to make one individual be subject to
additional scrutiny that no one else would be subject to. Please explain
why should one person be singled out for a special "six-month waiting
period" when the exact same possibility for conflict exists with anyone who
is ever on the committee?

Red herring. The objection (as many people have now voiced) is the double
standard you proposed.

Your selective quotation is impressive. Also in that interview, Linus
points out that his employment contract is probably "unique in the entire
world". Which means in practical terms that the details of what he has does
are fairly well irrelevant to any other situation. So what is the point in
bringing it up, at all, except to try and diminish someone else by
comparison?
(I'm done.)

Andrew Morton would be a prominent counter example in the Linux community.
He is employed by Google. Many other Linux developers are employed by Red
Hat, Intel, and other companies. At this point Linus relies on those people
for decisions on what goes into the kernel; it is much too big for a single
person to deal with even in review. I also expect the Linux community would
be overjoyed if every company provided drivers for their hardware. Subject
to review, of course.

Chuck

j***@gmail.com

2015-09-23 05:32:40 UTC

Of course, and the policies to deal with conflicts have deal with the
possibility that *anyone* at all might have conflict. But that was not your
suggestion. Your suggestion was to make one individual be subject to
additional scrutiny that no one else would be subject to. Please explain
why should one person be singled out for a special "six-month waiting
period" when the exact same possibility for conflict exists with anyone who
is ever on the committee?

I don't quite understand where the discussion went. The question was not
whether Travis is singled out but whether he is "singled in".

From my perspectives (7 to 8 years) the situation has changed a lot. Most
of the discussion and consensus building happens on github issues, pull
requests and mailing lists. Merge policy has changed a lot since the svn
days.

Based on all the comments, Travis doesn't have time for this. And I think
the final (last resort) decisions about code should be left to the active
developers that know and participate in the actual work.

If Travis is treated as developer but doesn't have a special status, then
there is also no reason to "single out" him and Continuum for possibly too
much influence.

This is already the current status quo as it developed over the last
several years, AFAICS.

In my view, in a narrow sense, Travis is a "hit and run" contributor. good
ideas and providing good contributions, but somebody has to integrate it,
maintain it and fit it into the development plan.
(Given my experience I would compare it more with GSOC contributions that
need the "core developers" to provide the continuity in the development to
absorb the good work.)
Travis has too many other obligations and interests to provide this day to
day continuity.

Travis is still very important for providing ideas, pushing projects
forward and as one of the community leaders, but I would leave the final
decisions for the development of numpy to the developers in the trenches.

I pretty much agree completely with Nathanial, and Sebastian, (except that
I don't know much about any other FOSS communities)

And to repeat my point from another thread on this: I'm very skeptical
about any committee or board that is based on "outside members" and that is
involved in the decision about code development.

Josef

Red herring. The objection (as many people have now voiced) is the double
standard you proposed.

Your selective quotation is impressive. Also in that interview, Linus
points out that his employment contract is probably "unique in the entire
world". Which means in practical terms that the details of what he has does
are fairly well irrelevant to any other situation. So what is the point in
bringing it up, at all, except to try and diminish someone else by
comparison?
(I'm done.)
Bryan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

Travis Oliphant

2015-09-22 23:12:33 UTC

Post by Stefan van der Walt
I guess we've gone off the rails pretty far at this point, so let me at
- I have never doubted that your intensions for NumPy are anything but
good (I know they are!),
- I *want* the community to be a welcoming place for companies to
contribute (otherwise, I guess I'd not be such a fervent supporter of
the scientific eco-system using the BSD license), and
- I love your enthusiasm for the project. After all, that is a big part
of what inspired me to become involved in the first place.
My goal is not to spread uncertainty, fear nor doubtâif that was the
perception left, I apologize.
I'll re-iterate that I wanted to highlight a concern about the
interactions of a (somewhat weakly cohesive) community and strong,
driven personalities such as yourself backed by a formidable amount of
development power. No matter how good your intensions are, there are
risks involved in this kind of interaction, and if we fail to even
*admit* that, we are in trouble.
Lest the above be read in a negative light again, let me state it
up-front: *I don't think you will hijack the project, use it for your
own gain, or attempt to do anything you don't believe to be in the best
interest of NumPy.* What I'm saying is that we absolutely need to move
forward in a way that brings everyone along, and makes everyone rest
assured that their voice will be heard.

Thank you for the clarification. I'm sorry that I started to question
your intentions. I agree that everyone should rest assured that their
voice will be heard. I have been and continue to be a staunch advocate
for the voices that are not even on this mailing list.

Post by Stefan van der Walt
Also, please know that I have not discussed these matters with Nathaniel
behind the scenes, other than an informal hour-long discussion about his
original governance proposal. There is no BIDS conspiracy or attempts
at crucifixion. After all, you were an invited guest speaker at an
event I organized this weekend, since I value your opinion and insights.

Thank you. I'm sorry for implying otherwise. That was wrong of me. I
know we are just trying to bring all the voices to the table.

Best,

-Travis

Fernando Perez

2015-09-23 08:02:16 UTC

Hi all,

I would like to pitch in here, I am sorry that I didn't have the time
before...

First, I want to disclose that recently Continuum made a research gift to
the Jupyter project; we were just now writing up a blog post to acknowledge
this, but in light of this discussion, I feel that I should say this up
front so folks can gauge any potential bias accordingly.

Travis, first, I'd like to kindly ask you not to conflate BIDS, an
institution where a large number of people work, with the personal opinions
of some, who happen to work there but in this case are speaking only for
themselves. You say "so many at BIDS", but as far as I know, your
disagreements are with Stefan and Nathaniel (Matthew doesn't work at
BIDS). You are painting with a very wide brush the work of many people,
and in the process, unfairly impacting others who have nothing to do with
this.

If anything, the only things I'm aware BIDS has done in an official
capacity regarding you or Continuum is to offer hosting for Continuum
developers at the DS4DS workshop and beyond (after an explicit request by
Matt Rocklin, and one we were delighted to honor), and our hosting of your
lecture in our Friday Data Science Lecture Series last week.

With that out of the way...

1. I hope the discussion can move past the suspicion and innuendo about
Continuum and Travis. I haven't always agreed with how Travis communicates
some of his ideas, and I've said it to him in such instances (e.g. this
weekend, as I myself was surprised at how his last round of comments had
landed on the list a few days back). But I also have worked closely with
him for years because I know that he has proven, not in words, but in
actions, that he has the best interests of our community at heart, and that
he is willing to try and do everything in his power to help whenever he
can.

When we founded Numfocus back in 2012, it would have been impossible for it
to really bootstrap without Travis' generosity, since he effectively footed
the bill for resources that were critically needed at the start. And yet,
he was always willing to take every step necessary to help Numfocus grow
independent of Continuum, so that it could be a real community asset:
today, there's not a single Continuum employee on the NF board (Travis and
I both resigned from the board a while back to allow for some fresh blood).

The creation and open-sourcing of conda has also been a critical
contribution, that I know many of us have benefited from: we all carry the
scars from the python packaging horror shows, and conda/anaconda has been a
life-changer. The fact that conda itself is open, means we have a core tool
that we can build upon.

To put it bluntly, few people in the whole world have given more of their
life, energy and resources to our community than Travis, and have done so
as generously as he has. He may have made mistakes, and again, I often
disagree with how he communicates. But accusations and innuendo like the
ones in this thread are damaging, hurtful and useless. And one thing that
I hope people will remember is that, famous and powerful as Travis may be,
he's still our colleague, a member of our community, and *a human being*,
so let's remember that as well...

2. Conflicts of interest are a fact of life, in fact, I would argue that
every healthy and sufficiently interconnected community eventually *should*
have conflicts of interest. They are a sign that there is activity across
multiple centers of interest, and individuals with connections in multiple
areas of the community. And we *want* folks who are engaged enough
precisely to have such interests!

For conflict of interest management, we don't need to reinvent the wheel,
this is actually something where our beloved institutions, blessed be their
bureaucratic souls, have tons of training materials that happen to be not
completely useless. Most universities and the national labs have
information on COIs that provides guidelines, and Numpy could include in
its governance model more explicit language about COIs if desired.

So, the issue is not to view COIs as something evil or undesirable, but
rather as the very real consequence of operating in an interconnected set
of institutions. And once you take that stance, you deal with that
rationally and realistically.

For example, you accept that companies aren't the only ones with potential
COIs: *all* entities have them. As Ryan May aptly pointed out, the notion
that academic institutions are somehow immune to hidden agendas or other
interests is naive at best... And I say that as someone who has happily
stayed in academia, resisting multiple overtures from industry over the
years, but not out of some quaint notion that academia is a pristine haven
of principled purity. Quite the opposite: in building large and complex
projects, I've seen painfully close how the university/government research
world has its own flavor of the same power, financial and political
ugliness that we attribute to the commercial side.

3. Commercial actors. Following up on the last paragraph, we should accept
that *all* institutions have agendas, not just companies. We live in a
world with companies, and I think it's naive to take a knee-jerk
anti-commercial stance: our community has had a productive and successful
history of interaction with industry in the past, and hopefully that will
continue in the future.

What is true, however, is that community projects should maintain the "seat
of power" in the community, and *not* in any single company. In fact, this
is important even to ensure that many companies feel comfortable engaging
the projects, precisely so they know that the technology is driven in an
open and neutral way even if some of their competitors participate.

That's why a governance model that is anchored in neutral ground is so
important. We've worked hard to make Numfocus the legal entity that can
play that role (that's why it's a 501(c)3), and that's why we've framed our
governance model for Jupyter in a way that makes all the institutions
(including Berkeley and Cal Poly) simply 'partners' that contribute by
virtue of supporting employees. But the owners of the decisions are the
*individuals* who do the work and form the community, not the
companies/institutions.

If we accept these premises, then hopefully we can have a rational
conversation about how to build a community, where at any point in time,
any of us should be judged on the merit of our actions, not the
hypotheticals of our intentions or our affiliations (commercial,
government, academic, etc).

Sorry for the long wall of text, I rarely post on this list anymore. But I
was saddened to see the turn of this thread, and I hope I can contribute
some perspective (and not make things worse :)

Cheers,

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Francesc Alted

2015-09-23 10:39:59 UTC

Hi Fernando,

I am happy that you decided to chime in here. This thread derailed in a
bad way and I hope that your wise words will help to redress the situation.

In fact, I would like to propose you having part of a future steering
committee of NumPy. I know that you may never have been implied in the
hard core development of NumPy, but my perception is that your opinions are
highly respected by almost everybody in the NumPy ecosystem. More than
that, you have this rare ability of being able to get both a donation from
Microsoft and at the same time (same year?) being awarded by the FSF, which
frankly, is not an easy thing to do ;)

Just to clear, I am not saying that you should act as the person for
deciding the roadmap for NumPy at all, but a person that should be in
charge of acting as an independent referee in the foreseeable Conflicts of
Interest in the NumPy roadmap.

Sorry if that means more work for you Fernando, because I know that you
have become a very busy person, but I also know how much do you care about
the NumPy ecosystem, and IMHO the NumPy community needs a person like you
*now*.

Francesc

Post by Fernando Perez
Hi all,
I would like to pitch in here, I am sorry that I didn't have the time
before...
First, I want to disclose that recently Continuum made a research gift to
the Jupyter project; we were just now writing up a blog post to acknowledge
this, but in light of this discussion, I feel that I should say this up
front so folks can gauge any potential bias accordingly.

Travis, first, I'd like to kindly ask you not to conflate BIDS, an
institution where a large number of people work, with the personal opinions
of some, who happen to work there but in this case are speaking only for
themselves. You say "so many at BIDS", but as far as I know, your
disagreements are with Stefan and Nathaniel (Matthew doesn't work at
BIDS). You are painting with a very wide brush the work of many people,
and in the process, unfairly impacting others who have nothing to do with
this.
If anything, the only things I'm aware BIDS has done in an official
capacity regarding you or Continuum is to offer hosting for Continuum
developers at the DS4DS workshop and beyond (after an explicit request by
Matt Rocklin, and one we were delighted to honor), and our hosting of your
lecture in our Friday Data Science Lecture Series last week.
With that out of the way...
1. I hope the discussion can move past the suspicion and innuendo about
Continuum and Travis. I haven't always agreed with how Travis communicates
some of his ideas, and I've said it to him in such instances (e.g. this
weekend, as I myself was surprised at how his last round of comments had
landed on the list a few days back). But I also have worked closely with
him for years because I know that he has proven, not in words, but in
actions, that he has the best interests of our community at heart, and that
he is willing to try and do everything in his power to help whenever he
can.
When we founded Numfocus back in 2012, it would have been impossible for
it to really bootstrap without Travis' generosity, since he effectively
footed the bill for resources that were critically needed at the start. And
yet, he was always willing to take every step necessary to help Numfocus
today, there's not a single Continuum employee on the NF board (Travis and
I both resigned from the board a while back to allow for some fresh blood).
The creation and open-sourcing of conda has also been a critical
contribution, that I know many of us have benefited from: we all carry the
scars from the python packaging horror shows, and conda/anaconda has been a
life-changer. The fact that conda itself is open, means we have a core tool
that we can build upon.
To put it bluntly, few people in the whole world have given more of their
life, energy and resources to our community than Travis, and have done so
as generously as he has. He may have made mistakes, and again, I often
disagree with how he communicates. But accusations and innuendo like the
ones in this thread are damaging, hurtful and useless. And one thing that
I hope people will remember is that, famous and powerful as Travis may be,
he's still our colleague, a member of our community, and *a human being*,
so let's remember that as well...
2. Conflicts of interest are a fact of life, in fact, I would argue that
every healthy and sufficiently interconnected community eventually *should*
have conflicts of interest. They are a sign that there is activity across
multiple centers of interest, and individuals with connections in multiple
areas of the community. And we *want* folks who are engaged enough
precisely to have such interests!
For conflict of interest management, we don't need to reinvent the wheel,
this is actually something where our beloved institutions, blessed be their
bureaucratic souls, have tons of training materials that happen to be not
completely useless. Most universities and the national labs have
information on COIs that provides guidelines, and Numpy could include in
its governance model more explicit language about COIs if desired.
So, the issue is not to view COIs as something evil or undesirable, but
rather as the very real consequence of operating in an interconnected set
of institutions. And once you take that stance, you deal with that
rationally and realistically.
For example, you accept that companies aren't the only ones with potential
COIs: *all* entities have them. As Ryan May aptly pointed out, the notion
that academic institutions are somehow immune to hidden agendas or other
interests is naive at best... And I say that as someone who has happily
stayed in academia, resisting multiple overtures from industry over the
years, but not out of some quaint notion that academia is a pristine haven
of principled purity. Quite the opposite: in building large and complex
projects, I've seen painfully close how the university/government research
world has its own flavor of the same power, financial and political
ugliness that we attribute to the commercial side.
3. Commercial actors. Following up on the last paragraph, we should
accept that *all* institutions have agendas, not just companies. We live
in a world with companies, and I think it's naive to take a knee-jerk
anti-commercial stance: our community has had a productive and successful
history of interaction with industry in the past, and hopefully that will
continue in the future.
What is true, however, is that community projects should maintain the
"seat of power" in the community, and *not* in any single company. In
fact, this is important even to ensure that many companies feel comfortable
engaging the projects, precisely so they know that the technology is driven
in an open and neutral way even if some of their competitors participate.
That's why a governance model that is anchored in neutral ground is so
important. We've worked hard to make Numfocus the legal entity that can
play that role (that's why it's a 501(c)3), and that's why we've framed our
governance model for Jupyter in a way that makes all the institutions
(including Berkeley and Cal Poly) simply 'partners' that contribute by
virtue of supporting employees. But the owners of the decisions are the
*individuals* who do the work and form the community, not the
companies/institutions.
If we accept these premises, then hopefully we can have a rational
conversation about how to build a community, where at any point in time,
any of us should be judged on the merit of our actions, not the
hypotheticals of our intentions or our affiliations (commercial,
government, academic, etc).
Sorry for the long wall of text, I rarely post on this list anymore. But
I was saddened to see the turn of this thread, and I hope I can contribute
some perspective (and not make things worse :)
Cheers,
--
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
Francesc Alted

Sebastian Berg

2015-09-23 13:55:48 UTC

Post by Francesc Alted
Hi Fernando,
I am happy that you decided to chime in here. This thread derailed in a
bad way and I hope that your wise words will help to redress the situation.
In fact, I would like to propose you having part of a future steering
committee of NumPy. I know that you may never have been implied in the
hard core development of NumPy, but my perception is that your opinions are
highly respected by almost everybody in the NumPy ecosystem. More than
that, you have this rare ability of being able to get both a donation from
Microsoft and at the same time (same year?) being awarded by the FSF, which
frankly, is not an easy thing to do ;)
Just to clear, I am not saying that you should act as the person for
deciding the roadmap for NumPy at all, but a person that should be in
charge of acting as an independent referee in the foreseeable Conflicts of
Interest in the NumPy roadmap.

Is it ok if I get a bit angry soon ;)? We can find many great community leaders. But please tell me that you all feel that numpy is so central or whatever that these community leaders should be in some sense in charge of numpy. I am ready to accept that and maybe it can be helpful, and a huge gain for numpy, but I would like to see a clear statement and reasons.

I like Fernandos mail, too, nor do I doubt Travis achievements. But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance. Now if you say:

Guys, you should have some community leader guidance in numpy, then we can discuss who. But to me it is not obvious that community leaders who are *not* directly active in numpy should be explicitely in governance. I am aware that everyone wants to help, but right now I do not feel helped at all :).

- Sebastian

Post by Francesc Alted
Sorry if that means more work for you Fernando, because I know that you
have become a very busy person, but I also know how much do you care about
the NumPy ecosystem, and IMHO the NumPy community needs a person like you
*now*.
Francesc

Travis, first, I'd like to kindly ask you not to conflate BIDS, an
institution where a large number of people work, with the personal opinions
of some, who happen to work there but in this case are speaking only for
themselves. You say "so many at BIDS", but as far as I know, your
disagreements are with Stefan and Nathaniel (Matthew doesn't work at
BIDS). You are painting with a very wide brush the work of many people,
and in the process, unfairly impacting others who have nothing to do with
this.
If anything, the only things I'm aware BIDS has done in an official
capacity regarding you or Continuum is to offer hosting for Continuum
developers at the DS4DS workshop and beyond (after an explicit request by
Matt Rocklin, and one we were delighted to honor), and our hosting of your
lecture in our Friday Data Science Lecture Series last week.
With that out of the way...
1. I hope the discussion can move past the suspicion and innuendo about
Continuum and Travis. I haven't always agreed with how Travis communicates
some of his ideas, and I've said it to him in such instances (e.g. this
weekend, as I myself was surprised at how his last round of comments had
landed on the list a few days back). But I also have worked closely with
him for years because I know that he has proven, not in words, but in
actions, that he has the best interests of our community at heart, and that
he is willing to try and do everything in his power to help whenever he
can.
When we founded Numfocus back in 2012, it would have been impossible for
it to really bootstrap without Travis' generosity, since he effectively
footed the bill for resources that were critically needed at the start. And
yet, he was always willing to take every step necessary to help Numfocus
today, there's not a single Continuum employee on the NF board (Travis and
I both resigned from the board a while back to allow for some fresh blood).
The creation and open-sourcing of conda has also been a critical
contribution, that I know many of us have benefited from: we all carry the
scars from the python packaging horror shows, and conda/anaconda has been a
life-changer. The fact that conda itself is open, means we have a core tool
that we can build upon.
To put it bluntly, few people in the whole world have given more of their
life, energy and resources to our community than Travis, and have done so
as generously as he has. He may have made mistakes, and again, I often
disagree with how he communicates. But accusations and innuendo like the
ones in this thread are damaging, hurtful and useless. And one thing that
I hope people will remember is that, famous and powerful as Travis may be,
he's still our colleague, a member of our community, and *a human being*,
so let's remember that as well...
2. Conflicts of interest are a fact of life, in fact, I would argue that
every healthy and sufficiently interconnected community eventually *should*
have conflicts of interest. They are a sign that there is activity across
multiple centers of interest, and individuals with connections in multiple
areas of the community. And we *want* folks who are engaged enough
precisely to have such interests!
For conflict of interest management, we don't need to reinvent the wheel,
this is actually something where our beloved institutions, blessed be their
bureaucratic souls, have tons of training materials that happen to be not
completely useless. Most universities and the national labs have
information on COIs that provides guidelines, and Numpy could include in
its governance model more explicit language about COIs if desired.
So, the issue is not to view COIs as something evil or undesirable, but
rather as the very real consequence of operating in an interconnected set
of institutions. And once you take that stance, you deal with that
rationally and realistically.
For example, you accept that companies aren't the only ones with potential
COIs: *all* entities have them. As Ryan May aptly pointed out, the notion
that academic institutions are somehow immune to hidden agendas or other
interests is naive at best... And I say that as someone who has happily
stayed in academia, resisting multiple overtures from industry over the
years, but not out of some quaint notion that academia is a pristine haven
of principled purity. Quite the opposite: in building large and complex
projects, I've seen painfully close how the university/government research
world has its own flavor of the same power, financial and political
ugliness that we attribute to the commercial side.
3. Commercial actors. Following up on the last paragraph, we should
accept that *all* institutions have agendas, not just companies. We live
in a world with companies, and I think it's naive to take a knee-jerk
anti-commercial stance: our community has had a productive and successful
history of interaction with industry in the past, and hopefully that will
continue in the future.
What is true, however, is that community projects should maintain the
"seat of power" in the community, and *not* in any single company. In
fact, this is important even to ensure that many companies feel comfortable
engaging the projects, precisely so they know that the technology is driven
in an open and neutral way even if some of their competitors participate.
That's why a governance model that is anchored in neutral ground is so
important. We've worked hard to make Numfocus the legal entity that can
play that role (that's why it's a 501(c)3), and that's why we've framed our
governance model for Jupyter in a way that makes all the institutions
(including Berkeley and Cal Poly) simply 'partners' that contribute by
virtue of supporting employees. But the owners of the decisions are the
*individuals* who do the work and form the community, not the
companies/institutions.
If we accept these premises, then hopefully we can have a rational
conversation about how to build a community, where at any point in time,
any of us should be judged on the merit of our actions, not the
hypotheticals of our intentions or our affiliations (commercial,
government, academic, etc).
Sorry for the long wall of text, I rarely post on this list anymore. But
I was saddened to see the turn of this thread, and I hope I can contribute
some perspective (and not make things worse :)
Cheers,
--
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
Francesc Alted

Fernando Perez

2015-09-23 14:21:08 UTC

Post by Sebastian Berg
Is it ok if I get a bit angry soon ;)?

Don't worry, Sebastian :)

I appreciate Francesc's kind words, but I have no intention of imposing my
presence anywhere, it's not like I'm looking for extra work at this point.
The process of establishing governance has to come organically from within
a community.

Cheers

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Chris Barker - NOAA Federal

2015-09-23 15:47:21 UTC

Post by Sebastian Berg
But discussing who is great community leader, etc. is frankly not obvious to me related to numpy governance.

Thank you Sebastian.

Could we please try to get back to the governance issues, without
naming names? There are some specific questions on the table that need
to get hashed out.

Numpy does not have a BDFL. BDFLs are not selected, they are naturally
produced, and there is no one that fits that bill now. We *could*
decide to have a single individual executive leader, selected by some
sort of democratic process. But that is not the same as a BDFL. And I
think the almost-consensus now is to not have that.

So here is what I think is on the table:

We have the steering council. Which leaves two questions:

-How big should it be?
-Who will be on the original, "seed" council?

Note that as I understand the current draft of the governance doc,
once established, the council itself decides who is on the council. So
at this point we really are ONLY deciding how it's going to start. It
has to be bootstrapped somehow.

However, that had been contentious enough that it would probably be a
good idea to hash out some guidelines about the council membership.

Personally, I have no idea how big the council should be. Too big, and
there is no point, consensus is harder to reach the larger the group,
and the main (only?) role of the council is to resolve issues where
consensus has not been reached in the larger community. But what is
too big?

As for make-up of the council, I think we need to expand beyond people
who have recently contributed core code.

Yes, the council does need to have expertise to make technical
decisions, but if you think about the likely contentious issues like
ABI breakage, a core-code focused view is incomplete. So there should
be representation by:

Someone(s) with a long history of working with the code -- that
institutional memory of why decisions were made the way they were
could be key.

Someone(s) that may not have worked on the core code, but is a major
player in the community, perhaps as the leader of a Numpy-dependent
project. This would provide representation for the broad community.

I do want to note that the governance document as I understand it is
consistent with these suggestions.

As for conflict of interest issues, etc:

Chill out people!

Numpy is an open source project, if it gets hijacked, it can be forked.

And the council is also democratic -- no one person can drive the
project. If a council member is not acting in the interests of the
community, s/he can be removed.

NOTE: while x.org, and egcs, Xemacs, and ... may be examples of
failures of governance, they are also examples of successes of open
source.

-Chris

Charles R Harris

2015-09-23 17:12:31 UTC

On Wed, Sep 23, 2015 at 9:47 AM, Chris Barker - NOAA Federal <

Post by Sebastian Berg
But discussing who is great community leader, etc. is frankly not

obvious to me related to numpy governance.
Thank you Sebastian.
Could we please try to get back to the governance issues, without
naming names? There are some specific questions on the table that need
to get hashed out.
Numpy does not have a BDFL. BDFLs are not selected, they are naturally
produced, and there is no one that fits that bill now. We *could*
decide to have a single individual executive leader, selected by some
sort of democratic process. But that is not the same as a BDFL. And I
think the almost-consensus now is to not have that.
-How big should it be?
-Who will be on the original, "seed" council?
Note that as I understand the current draft of the governance doc,
once established, the council itself decides who is on the council. So
at this point we really are ONLY deciding how it's going to start. It
has to be bootstrapped somehow.
However, that had been contentious enough that it would probably be a
good idea to hash out some guidelines about the council membership.
Personally, I have no idea how big the council should be. Too big, and
there is no point, consensus is harder to reach the larger the group,
and the main (only?) role of the council is to resolve issues where
consensus has not been reached in the larger community. But what is
too big?
As for make-up of the council, I think we need to expand beyond people
who have recently contributed core code.
Yes, the council does need to have expertise to make technical
decisions, but if you think about the likely contentious issues like
ABI breakage, a core-code focused view is incomplete. So there should
Someone(s) with a long history of working with the code -- that
institutional memory of why decisions were made the way they were
could be key.
Someone(s) that may not have worked on the core code, but is a major
player in the community, perhaps as the leader of a Numpy-dependent
project. This would provide representation for the broad community.
I do want to note that the governance document as I understand it is
consistent with these suggestions.
Chill out people!
Numpy is an open source project, if it gets hijacked, it can be forked.
And the council is also democratic -- no one person can drive the
project. If a council member is not acting in the interests of the
community, s/he can be removed.

Hear, hear. Well put Chris. I don't disagree with any of this.

Post by Sebastian Berg
NOTE: while x.org, and egcs, Xemacs, and ... may be examples of
failures of governance, they are also examples of successes of open
source.

Good point.

Chuck

Travis Oliphant

2015-09-23 19:18:39 UTC

I accept that criticism and apologize for doing that. My *human* side was
coming out, and I was not being fair. In my head, though I was also
trying to illustrate how some seemed to be doing the same thing for
Continuum or other companies. This did not come out very artfully in the
early morning hours. I'm sorry. BIDS is doing a lot for the
community --- the recent DS4DS workshop, for example, was a spectacularly
useful summit --- I hope that many different write-ups and reports of the
event make their way out into the world.

Post by Fernando Perez
1. I hope the discussion can move past the suspicion and innuendo about
Continuum and Travis. I haven't always agreed with how Travis communicates
some of his ideas, and I've said it to him in such instances (e.g. this
weekend, as I myself was surprised at how his last round of comments had
landed on the list a few days back). But I also have worked closely with
him for years because I know that he has proven, not in words, but in
actions, that he has the best interests of our community at heart, and that
he is willing to try and do everything in his power to help whenever he
can.

I really hope it's just a perception problem (perhaps on my end). There
are challenges with working in the commercial world (there are a lot of
things to do that have nothing to do with the technology creation) and
communicating on open-source mailing lists. As many have noticed,
despite my intentions to contribute, I really can't do the same level of
contribution personally that I could when I was a student and a professor
and had more time.

However, I think that it is also under-appreciated (or mis-understood) how
much time I have spent with training and helping people who have
contributed instead. It's important to me to build a company that can
sponsor people to work on open-source (in a community setting). We are
still working on that, but it has been my intent. So, far it's actually
easier to sponsor new projects than it is to sponsor people on old
projects. I am quite sure that if Continuum had put 3 people full time on
NumPy in 2012, there would have been a lot of back-lash and
mis-understanding. That's why we didn't do it. The collateral effect
of that was the creation of other tools that could be somewhat competitive
with NumPy long term -- or not.

I'd like to learn how to work with the community in an optimal way so that
everyone benefits --- and progress happens. That's also why we created
Numfocus --- though it is ironic that NumPy has been one of the last
projects to actually sign up and be a formally sponsored project.

2. Conflicts of interest are a fact of life, in fact, I would argue that

Post by Fernando Perez
every healthy and sufficiently interconnected community eventually *should*
have conflicts of interest. They are a sign that there is activity across
multiple centers of interest, and individuals with connections in multiple
areas of the community. And we *want* folks who are engaged enough
precisely to have such interests!
For conflict of interest management, we don't need to reinvent the wheel,
this is actually something where our beloved institutions, blessed be their
bureaucratic souls, have tons of training materials that happen to be not
completely useless. Most universities and the national labs have
information on COIs that provides guidelines, and Numpy could include in
its governance model more explicit language about COIs if desired.
So, the issue is not to view COIs as something evil or undesirable, but
rather as the very real consequence of operating in an interconnected set
of institutions. And once you take that stance, you deal with that
rationally and realistically.
For example, you accept that companies aren't the only ones with potential
COIs: *all* entities have them. As Ryan May aptly pointed out, the notion
that academic institutions are somehow immune to hidden agendas or other
interests is naive at best... And I say that as someone who has happily
stayed in academia, resisting multiple overtures from industry over the
years, but not out of some quaint notion that academia is a pristine haven
of principled purity. Quite the opposite: in building large and complex
projects, I've seen painfully close how the university/government research
world has its own flavor of the same power, financial and political
ugliness that we attribute to the commercial side.

Thanks for re-emphasizing this. I agree it's O.K. and even necessary to
talk about COIs. If I have not been upfront about any possible COIs it's
not because of a desire to hide them -- I may not be aware of them myself
as I'm still focused on what technically can be better, and how to create a
company that can produce as much relevant open source software as possible.

I appreciate your helping us get to a better conversational ground.
Stefan's and Nathaniel's recent emails have done that as well.

Thank you,

-Travis

Fernando Perez

2015-09-23 20:38:31 UTC

Post by Travis Oliphant
I accept that criticism and apologize for doing that. My *human* side
was coming out, and I was not being fair. In my head, though I was
also trying to illustrate how some seemed to be doing the same thing for
Continuum or other companies. This did not come out very artfully in the
early morning hours. I'm sorry.

Thank you.

Post by Travis Oliphant
BIDS is doing a lot for the community --- the recent DS4DS workshop,
for example, was a spectacularly useful summit --- I hope that many
different write-ups and reports of the event make their way out into the
world.

We will put out a full public report, along with all the slides, which
we're collecting, etc. It may take us a few days, as we have an upcoming
event for BIDS and our two partner sites (UW and NYU), but we'll do it, as
we hope this will be useful to the entire community.

Cheers,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Matthew Brett

2015-09-23 19:57:19 UTC

Hi,

On Wed, Sep 23, 2015 at 1:02 AM, Fernando Perez <***@gmail.com> wrote:

[snip]

Post by Fernando Perez
1. I hope the discussion can move past the suspicion and innuendo about
Continuum and Travis

I'm glad the discussion has become a little more calm now, but I find
it difficult not to be annoyed by this statement above.

I see a severe reaction to perceived 'suspicion and innuendo', but I
see no 'suspicion and innuendo'. Unless you mean that any suggestion
of potential conflict of interest is suspicion and innuendo.

You imply rudeness and bad faith when you say this. I suppose this
must be aimed at me or Stefan or both of us. In any case, I think the
accusation is unhelpful and unfair. It makes this discussion more
difficult to have in the future.

See you,

Matthew

Fernando Perez

2015-09-23 20:36:35 UTC

Post by Matthew Brett
I see a severe reaction to perceived 'suspicion and innuendo', but I
see no 'suspicion and innuendo'. Unless you mean that any suggestion
of potential conflict of interest is suspicion and innuendo.

No, as I said, COIs are absolutely a fact of life, and *should* be talked
about, openly and directly. I was referring generically about the tone of
this thread, that Ryan described as "bizarre", others as "surpised",
"disheartened", etc.

Post by Matthew Brett
You imply rudeness and bad faith when you say this. I suppose this
must be aimed at me or Stefan or both of us. In any case, I think the
accusation is unhelpful and unfair. It makes this discussion more
difficult to have in the future.

It was sincerely my intention to frame my critique in a specific way that
made the discussion *easier* to have in the future, precisely by
acknowledging that COIs were part of life, and not only for commercial
entities.

I was NOT implying that Stefan and you were somehow guilty of anything, I
only mentioned your names when I asked Travis not to paint Berkeley folks
with a broad brush, that's all.

If only the two of you took offense at my wording, in the interest of
keeping this already contentious and fraught thread contained, I offer:

a) a public apology for my choice of words, since it was certainly not my
intent to offend you, and I was not aiming at you personally.

b) a suggestion that we discuss it further personally, taking advantage of
the fact that we happen to be physically close.

If anyone else would like me to answer in public because they feel a slight
on my part, I will do so.

Best,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Stefan van der Walt

2015-09-23 21:31:16 UTC

What I thought Matthew was saying is that you writing "I hope the
discussion can move past the suspicion and innuendo about Continuum and
Travis" gives validity to the view that those things are real whereas,
in my view, they were constructed in responses by others who didn't
accurately summarize the intent of what was being said (and I'm not
laying blame to anyone, I could certainly have worded myself more
clearly).

But, yes, it was disheartening—especially because we all know one
another and have hacked together late into the night at SciPy
conferences. My experience has always been that we are reasonable
people who listen; but perhaps we forget that about one another from
time to time.

It is important to me that we work towards making this mailing list a
safe place for discussion again. Part of that may be to do some
maintenance on bridges of trust that seem to have eroded a bit over the
years.

Perhaps some kind of group bonding activity, such as working on a shared
project, would help? ;)

Post by Fernando Perez
b) a suggestion that we discuss it further personally, taking advantage of
the fact that we happen to be physically close.

Sure, I'm happy to discuss the personal side of this offline.

Stéfan

Chris Barker

2015-09-23 21:44:29 UTC

Post by Fernando Perez
b) a suggestion that we discuss it further personally, taking advantage

Post by Fernando Perez
the fact that we happen to be physically close.

Sure, I'm happy to discuss the personal side of this offline.

Hey! I want to have a beer with you folks too! Maybe time for a trip back
to the old stomping grounds....

-Chris
--
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R (206) 526-6959 voice
7600 Sand Point Way NE (206) 526-6329 fax
Seattle, WA 98115 (206) 526-6317 main reception

***@noaa.gov

Fernando Perez

2015-09-23 21:50:53 UTC

Post by Stefan van der Walt
Perhaps some kind of group bonding activity, such as working on a shared
project, would help? ;)

Indeed, there's a bunch of fresh 2x4s, screws and bolts with your name on
them ;)

Cheers,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Matthew Brett

2015-09-23 21:32:27 UTC

Hi,

Sure, I said above, it is clearly true the some people reacted very strongly.

Did you see any remark made by me or Stefan or anyone else that could
reasonably be described as bizarre, surprising or disheartening?

Cheers,

Matthew

Charles R Harris

2015-09-23 21:51:50 UTC

Post by Matthew Brett
Hi,

No, as I said, COIs are absolutely a fact of life, and *should* be talked
about, openly and directly. I was referring generically about the tone

Post by Fernando Perez
this thread, that Ryan described as "bizarre", others as "surpised",
"disheartened", etc.

Sure, I said above, it is clearly true the some people reacted very strongly.
Did you see any remark made by me or Stefan or anyone else that could
reasonably be described as bizarre, surprising or disheartening?

Stephan and Matthew, go to your rooms ;)

Chuck

Fernando Perez

2015-09-23 22:08:50 UTC

Post by Matthew Brett
Did you see any remark made by me or Stefan or anyone else that could
reasonably be described as bizarre, surprising or disheartening?

As I said already, I wasn't referring to you personally, but to the tone of
the thread. It was the taste that the thread left in my mouth after
reading tens of very long emails, and I guess I wasn't the only one, since
multiple folks used such adjectives. But it was a poor choice of words on
my part.

And no, I didn't make a quote-by-quote analysis of the thread, I'm sorry.

One last time, it was *not* a personal reference to you: the only reason I
mentioned your names was because of the Berkeley clarification regarding
BIDS that I asked of Travis, that's all. If that comment hadn't been made,
I would not have made any mention whatsoever of anyone in particular. I
apologize for not foreseeing that this would have made you feel singled
out, in retrospect, I should have.

In my mind, it was the opposite, as I felt that you had every right to
express whatever opinions you have speaking for yourselves, independent of
your affiliations, and I was simply asking Travis to separate individuals
from institutions. But I should have realized that calling anyone out by
name in a context like this is a bad idea regardless.

Cheers,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Travis Oliphant

2015-09-23 22:19:49 UTC

Post by Fernando Perez
One last time, it was *not* a personal reference to you: the only reason I
mentioned your names was because of the Berkeley clarification regarding
BIDS that I asked of Travis, that's all. If that comment hadn't been made,
I would not have made any mention whatsoever of anyone in particular. I
apologize for not foreseeing that this would have made you feel singled
out, in retrospect, I should have.
In my mind, it was the opposite, as I felt that you had every right to
express whatever opinions you have speaking for yourselves, independent of
your affiliations, and I was simply asking Travis to separate individuals
from institutions. But I should have realized that calling anyone out by
name in a context like this is a bad idea regardless.

This was my fault for not being more careful in my words. I felt multiple
things when I wrote my emails that led to incorrectly chosen words --- but
mostly I was feeling unappreciated, attacked, and accused. I'm sure now
that was not intended --- but there have been mis-understandings. I expect
they will happen again. I know if we listen to each other and trust that
while we may see the world differently and have different framings of
solutions --- we can work to coordinate on an important technical activity
together.

In retrospect, my initial email requesting inclusion on the seed council
could have been worded better (as there were multiple things conflated
together). I am responding to the actual text of the governance document
in the other thread so as to clarify what my proposal actually is in the
context of that document.

-Travis

Fernando Perez

2015-09-23 22:37:45 UTC

Post by Travis Oliphant
This was my fault for not being more careful in my words. I felt
multiple things when I wrote my emails that led to incorrectly chosen words
--- but mostly I was feeling unappreciated, attacked, and accused. I'm
sure now that was not intended --- but there have been mis-understandings.
I expect they will happen again. I know if we listen to each other and
trust that while we may see the world differently and have different
framings of solutions --- we can work to coordinate on an important
technical activity together.
In retrospect, my initial email requesting inclusion on the seed council
could have been worded better (as there were multiple things conflated
together). I am responding to the actual text of the governance document
in the other thread so as to clarify what my proposal actually is in the
context of that document.

No worries, I wasn't digging it up further, really! Just clarifying for
other reasons, not digging into you :)

Glad to see the other thread proceeding further, let's let this one die as
peacefully as it can...

Best,

f

--
Fernando Perez (@fperez_org; http://fperez.org)
fperez.net-at-gmail: mailing lists only (I ignore this when swamped!)
fernando.perez-at-berkeley: contact me here for any direct mail

Paul Ivanov

2015-09-24 05:06:53 UTC

Post by Fernando Perez
Glad to see the other thread proceeding further, let's let this one die as
peacefully as it can...

This Monty Python sketch seems relevant:

(hope everyone's in good health and regains better spirits, back to lurking
for me)
--
_
/ \
A* \^ -
,./ _.`\\ / \
/ ,--.S \/ \
/ `"~,_ \ \
__o ?
_ \<,_ /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7

Matthew Brett

2015-09-23 22:29:17 UTC

Post by Matthew Brett
Did you see any remark made by me or Stefan or anyone else that could
reasonably be described as bizarre, surprising or disheartening?

You know me well, I not reacting to personal offense, otherwise I
would email you privately.

See you,

Matthew

Travis Oliphant

2015-09-22 06:29:12 UTC

A couple of things to help clarify:

1) nobody believes that the community should be forced to adopt numba as
part of ufunc core yet --- but this could happen someday just as Cython is
now being adopted but was proposed 8 years ago that it "could be adopted"
That's a red-hearing.

2) I have stated that breaking the ABI is of little consequence because
of conda as well as other tools. I still believe that. This has nothing
to do with any benefit Continuum might or might not receive because of
conda. Everyone else who wants to make a conda-based distribution also
benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
I don't think the community realizes the damange that is done with FUD like
this. There are real implications. It halts progress, creates confusion,
and I think ultimately damages the community.

Numba being an acceptable dependency means a lot more than conda --- it's
dependent on LLVM compiled support which would have to be carefully tested
--- first as only an optional dependency for many years.

I don't see how. None of these have been proposed for integrating into
NumPy. I don't see how integrating numba into NumPy benefits Continuum
at all. It's much easier for us to keep it separate. At this point
Continuum doesn't have an opinion about integrating DyND into NumPy or not.

These projects will all succeed or fail on their own based on users needs.
Whether or not they every become a part of NumPy will depend on whether
they are useful as such not because a person at Continuum is part of a
steering committee (with other people on it).

I know that you were responding to specific question by Brian as to how
their could be a conflict of interest for Continuum and NumPy development.
I don't think this is a useful conversation --- we could dream up all
kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
really wants Spark to take over and for NumPy to have special connections
to Spark). Are we to not allow anyone at BIDS to participate in the
steering council because of their other interests?

But remember, the original point is whether or not someone from Continuum
(or I presume any company and not just singling out Continuum for special
treatment) should be on the steering council. Are you really arguing
that they shouldn't because there are other projects Continuum is working
on that have some overlap with NumPy. I really hope you don't actually
believe that.

-Travis

Post by Stefan van der Walt
StÃ©fan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Stefan van der Walt

2015-09-22 07:16:03 UTC

Hi Travis

Post by Travis Oliphant
1) nobody believes that the community should be forced to adopt numba as
part of ufunc core yet --- but this could happen someday just as Cython is
now being adopted but was proposed 8 years ago that it "could be adopted"
That's a red-hearing.

Yes, I'd like to clarify: I was not against including any specific
technology in NumPy. I was highlighting that there may be different
motivations for members of the general community and those working for,
say, Continuum, to get certain features adopted.

Post by Travis Oliphant
2) I have stated that breaking the ABI is of little consequence because
of conda as well as other tools. I still believe that. This has nothing
to do with any benefit Continuum might or might not receive because of
conda. Everyone else who wants to make a conda-based distribution also
benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
I don't think the community realizes the damange that is done with FUD like
this. There are real implications. It halts progress, creates confusion,
and I think ultimately damages the community.

This is an old argument, and the reason why we have extensive measures
in place to guard against ABI breakage. But, reading what you wrote
above, I would like to understand better what FUD you are referring to,
because I, rightly or wrongly, believe there is a real concern here that
is being glossed over.

Post by Travis Oliphant
I don't see how. None of these have been proposed for integrating into
NumPy. I don't see how integrating numba into NumPy benefits Continuum
at all. It's much easier for us to keep it separate. At this point
Continuum doesn't have an opinion about integrating DyND into NumPy or not.

I think that touches, tangentially at least, on the problem. If an
employee of Continuum were steering NumPy, and the company developed an
opinion on those integrations, would such a person not feel compelled to
toe the company line? (Whether the company is Continuum or another is
besides the point—I am only trying to understand the dynamics of working
for a company and leading an open source project that closely interacts
with their producs.)

Post by Travis Oliphant
I know that you were responding to specific question by Brian as to how
their could be a conflict of interest for Continuum and NumPy development.
I don't think this is a useful conversation --- we could dream up all
kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
really wants Spark to take over and for NumPy to have special connections
to Spark). Are we to not allow anyone at BIDS to participate in the
steering council because of their other interests?

I guess that's an interesting example, but BIDS (which sits inside a
university and is funded primarily by foundations) has no financial, and
very few other, incentives to do so.

Post by Travis Oliphant
But remember, the original point is whether or not someone from Continuum
(or I presume any company and not just singling out Continuum for special
treatment) should be on the steering council. Are you really arguing
that they shouldn't because there are other projects Continuum is working
on that have some overlap with NumPy. I really hope you don't actually
believe that.

Here's what I'm trying to say (and I apologise for ruffling feathers in
the process):

There are concerns amongst members of the community that (will) arise
when strong players from industry try / hint at exerting (some)
executive control over NumPy. We can say that these concerns amount to
spreading FUD, that they are uninformed, unrealistic, etc., but
ultimately they are still out there, and until they are discussed and
addressed, I find it hard to see how we can move forward with ease.

Stéfan

Travis Oliphant

2015-09-22 09:33:24 UTC

Post by Matthew Brett
Hi Travis

Post by Travis Oliphant
1) nobody believes that the community should be forced to adopt numba

Post by Travis Oliphant
part of ufunc core yet --- but this could happen someday just as Cython

Post by Travis Oliphant
now being adopted but was proposed 8 years ago that it "could be adopted"
That's a red-hearing.

This is what I'm calling you out on. Why? I think that is an unfair
statement and inaccurate. The general community includes Continuum,
Enthought, Microsoft, Intel, various hedge funds, investment banks and
companies large and small. Are you saying that people should not be
upfront about their affiliations with a company? That if they are not
academics, then they should not participate in the discussion? It is hard
enough to be at a company and get time to contribute effort back to an open
source project. We should not be questioning people's motives just
*because* they are at a company. We should not assume people cannot think
in terns of the success of the project, just because they are at a company.

Their proposals and contributions can be evaluated on their merits and
value --- so this whole discussion seems to be just revealing an
anti-company paranoia rather than helping understand the actual concern.

Post by Travis Oliphant
2) I have stated that breaking the ABI is of little consequence because
of conda as well as other tools. I still believe that. This has

nothing

Post by Travis Oliphant
to do with any benefit Continuum might or might not receive because of
conda. Everyone else who wants to make a conda-based distribution also
benefits (Cloudera, Microsoft, Intel, ...) or use conda also benefits.
I don't think the community realizes the damange that is done with FUD

Post by Travis Oliphant
this. There are real implications. It halts progress, creates

confusion,

Post by Travis Oliphant
and I think ultimately damages the community.

This is an old argument, and the reason why we have extensive measures
above, I would like to understand better what FUD you are referring to,
because I, rightly or wrongly, believe there is a real concern here that
is being glossed over.

I don't know which is the "old argument". Anyway, old arguments can still
be right. The fact is that not breaking the ABI has caused real damage to
the community. NumPy was never designed to not have it's ABI broken for
over a decade. We have some attempts to guard against ABI breakage ---
but they are not perfect.

We have not moved the code-base forward for fear of breaking the ABI.
When it was hard to update your Python installation that was a concern.
There are very few cases where this is still the concern (conda is a big
part of it but not the only part as other distros and approaches for easily
updating the install exist) --- having this drive major architecture
decisions is a serious mistake in my mind, and causes a lot more work than
it should.

The FUD I'm talking about is the anti-company FUD that has influenced
discussions in the past. I really hope that we can move past this.

I think that touches, tangentially at least, on the problem. If an
employee of Continuum were steering NumPy, and the company developed an
opinion on those integrations, would such a person not feel compelled to
toe the company line? (Whether the company is Continuum or another is
besides the pointâI am only trying to understand the dynamics of working
for a company and leading an open source project that closely interacts
with their producs.)

O.K. if you are honestly asking this question out of inexperience, then I
can at least help you understand because perhaps that is the problem
(creating a straw-man that doesn't exist). I have never seen a motivated
open source developer at a company who "tows the company line" within a
community project that is accepted long term. All that would do is drive
the developer out of the company and be a sure-fire way to make sure their
contributions are not accepted. I know that at Continuum, for example, if
someone disagreed with me about an open source technology, didn't tell me
and just "towed the company line" (whatever they thought that was) --- they
would be fired. Most software companies that participate in open source
are like that. The worst thing that happens is the company no longer
participates in the discussion --- that is what happens if they can't get
contributions.

It really is no different than me being worried that an academic might push
to get something into a library because it would "help their publication
record" --- which I also think is a very real potential concern. Or
perhaps you have a favor owed to another colleague who helped you along in
your academic career and they "really want" some feature added. The
actual conflict of interest that exists there has the same amount of weight
in my mind as the ones you hypothesize about.

Post by Travis Oliphant
I know that you were responding to specific question by Brian as to how
their could be a conflict of interest for Continuum and NumPy

development.

Post by Travis Oliphant
I don't think this is a useful conversation --- we could dream up all
kinds of conflicts of interest for BIDS and NumPy too (e.g. perhaps BIDS
really wants Spark to take over and for NumPy to have special connections
to Spark). Are we to not allow anyone at BIDS to participate in the
steering council because of their other interests?

I guess that's an interesting example, but BIDS (which sits inside a
university and is funded primarily by foundations) has no financial, and
very few other, incentives to do so.

I think you don't understand how academic finances work. I could see a
lot of ways such interest could develop --- and could influence funding
agencies. I don't think they are very likely --- but they are possible
--- and to me the same degree of possibility that Continuum would try to
"force anything" on the NumPy community.

Here's what I'm trying to say (and I apologise for ruffling feathers in
There are concerns amongst members of the community that (will) arise
when strong players from industry try / hint at exerting (some)
executive control over NumPy. We can say that these concerns amount to
spreading FUD, that they are uninformed, unrealistic, etc., but
ultimately they are still out there, and until they are discussed and
addressed, I find it hard to see how we can move forward with ease.

I'm sorry you have these concerns. I don't think they are as warranted
as you believe. NumPy will get strong players from industry coming in.
In fact, one of the reasons we all felt it was important to establish
Numfocus was to be an organization that could shield these players from the
development of NumPy and other projects.

I am not one of those "strong industry players" you speak of. I am a
friend. I am a participant. I am one of the community. Perhaps you and
others feel that *I* am that strong industry player trying to exert
"executive control" over NumPy. Is that true? Is that what you feel?
I'm starting to wonder if that is actually the problem here.

-Travis

Post by Matthew Brett
StÃ©fan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Stephan Hoyer

2015-09-22 10:43:07 UTC

Post by Travis Oliphant
The FUD I'm talking about is the anti-company FUD that has influenced
discussions in the past. I really hope that we can move past this.

I have mostly stayed out of the governance discussion, in deference to how
new I am in this community, but I do want to take a moment to speak up here
to echo Travis's concern about anti-company FUD.

Everyone invested in NumPy has their own projects, priorities and employers
which shape their agenda. As far as I can tell, Travis and Continuum have
only ever acted with the long term health of the scipy ecosystem in mind.

Stephan

Nathaniel Smith

2015-09-22 08:12:51 UTC

Post by Bryan Van de Ven
There is ample history of such things happening in OSS history, so I think that's a fair concern, even if that has not happened for numpy yet.

I have no expectation that continuum will follow any of these paths,
and in most cases am not even sure what that would mean, BUT just
because I think it is useful to have a wide variety of concrete
examples to draw on -- data is good! -- there actually are *lots* of
examples of "community revolts" wresting projects from their original
founders, in a variety of corporate and non-corporate contexts. Some
examples include the nodejs->iojs fork and merge (which was about
wresting control of the project from the founding company), the
gcc->egcs fork and merge (which removed RMS's control over day-to-day
running of the project), the openoffice->libreoffice fork, the
xfree86->x.org fork (where the original core team decided to change
the license and all the developers left), the mambo->joomla fork, the
xchat->hexchat fork (triggered partially by people's annoyance at the
original developer for trying to monetize the project), ... Along
somewhat similar lines, there's also the fraught history of Qt and
Trolltech and the conflicts between the community and commercial
interests there.

-n

--
Nathaniel J. Smith -- http://vorpus.org

Bryan Van de Ven

2015-09-22 08:21:56 UTC

Post by Nathaniel Smith
I have no expectation that continuum will follow any of these paths,
and in most cases am not even sure what that would mean, BUT just
because I think it is useful to have a wide variety of concrete
examples to draw on -- data is good! -- there actually are *lots* of
examples of "community revolts" wresting projects from their original
founders, in a variety of corporate and non-corporate contexts. Some
examples include the nodejs->iojs fork and merge (which was about
wresting control of the project from the founding company), the
gcc->egcs fork and merge (which removed RMS's control over day-to-day
running of the project), the openoffice->libreoffice fork, the
xfree86->x.org fork (where the original core team decided to change
the license and all the developers left), the mambo->joomla fork, the
xchat->hexchat fork (triggered partially by people's annoyance at the
original developer for trying to monetize the project), ... Along
somewhat similar lines, there's also the fraught history of Qt and
Trolltech and the conflicts between the community and commercial
interests there.

All of there are exactly the opposite of what I asked about, and what was suggested as the threat: an original founder and corporate interest wresting control from the community.

Bryan

Travis Oliphant

2015-09-22 05:01:51 UTC

Post by Matthew Brett
Hi Travis, and all,
You might have seen I was advocating for having someone who takes
final responsibility for the project, partly to get discussions
unstuck, as you said.
I agree with Chris, that at this stage, there is no-one who could be
Benevolent Dictator for the project. It seems to me that (nearly)
everyone would agree that, if there is a leader, we would not want a
person who dictates, but someone who is good at putting in the hard
work and time it takes to hear all the arguments, and guide people to
agreement, where that is possible. If there is a leader, I think we
should select them for those skills.

I don't really agree that someone couldn't be found. I think the key is
that 1) the person would be able to make a decision about what needs to be
done if the community is stuck and 2) their is not really a "for life"
clause in that their would be a way to call for a replacement.

Post by Matthew Brett
For the proposal that you join the steering committee, I see two problems.
The first is, that the members of the steering committee have to be
the people who are keeping in touch with the day to day work of the
project. I am sure you would agree that, in the past, you have not
had enough time for that. Your recent emails about the new work you
want to do, also imply that you may be too busy to get involved in the
detailed discussions needed for getting the code merged. In any case,
I think you'd also agree that in the past you have hoped that you
would have more time for numpy than you did.

I don't think this is true. The steering committee would only be called
upon to unstick things and make decisions. At those times, it would not
take long to come up to speed on the issues and make a decision.

Post by Matthew Brett
The second problem is that you have a potential conflict of interest,
in that it is possible for the needs of Continuum to conflict with the
needs of numpy.

I think this is a red-herring and should not be an issue --- except for
obvious situations where I would have to not participate in a "vote" (such
as whether or not Continuum would be an institutional sponsor or something
like that).

Everyone has associations and affiliations and goals and plans that could
lead to conflicts of interest. This kind of requirement un-necessarily
limits the governance to only certain kinds of people who have only
"volunteer" time. This is actually quite damaging as it limits the
ability for people to get paid to work on NumPy. This feels like a
world-view debate that is actually best left to a different mailing list.

Post by Matthew Brett
I believe, from previous emails on this list, that
you don't think that is very important, but I continue to disagree
about that.

For example, see this interview with Linus Torvalds,

Post by Matthew Brett
where he talks about going out of his way to make sure that people can
trust his decisions, despite the fact he is paid to work on Linux [1].

But this is in a BDFL role and not a steering committee role with 9 other
members. I don't think the situation compares or applies at all. The
unwarranted fear this kind of reasoning can create in the mind of otherwise
reasonable people is unfortunate.

Post by Matthew Brett
In practice, the most obvious step that I can think of, is to defer
the decision as to whether you are on the steering committee for six
months. I guess over that time you will be working with the other
numpy developers more closely.

Fundamentally the "steering committee" is too big. I think the
committee should be much smaller (no more than 5 and really three is the
right size for what it is supposed to actually do). I've had experience
with small committees and large committees. Large councils make it very
difficult to move things forward.

If it is to remain that big, I think it needs people on it like me, Robert
Kern, or David Cournapeau who have a longer history with the project and
understand why some of the things are the way they are. I still hear far
too many conversations that start from a lack of understanding of the early
conversations that led to why NumPy is the way it is. This lack of
context is not helpful and potentially dangerous. The advanced indexing
discussions happening right now and the renewed array-scalar discussions
for example are both examples of features that have had previous
discussions and have a long history of back and forth between various
people.

Post by Matthew Brett
I should say that Stefan and I are working on a governance proposal,
in this case for scikit-image, where we split the governance into 1)
the developers doing the work and making the day to day decisions and
2) the trustees, usually from other relevant projects, or
no-longer-active developers, who make sure the project does not go off
the rails. I think you'd be an excellent and obvious trustee, in
that model.

I like the trustee model too and think such an addition to the NumPy
concept would help alleviate my concerns about actually being on a
"steering committee" but my preferred outcome is actually that the agreed
upon steering council be smaller and that people who have a right to vote
on things like the make-up of the steering committee be comprised of people
who have been significantly involved in the past 3 years (not just the past
one year).

-Travis

Post by Matthew Brett
Cheers,
Matthew
[1] http://www.bbc.com/news/technology-18419231
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Stefan van der Walt

2015-09-22 05:57:12 UTC

Post by Travis Oliphant
I would recommend three possible adjustments to the steering council
concept.
1 - define a BDFL for the council. I would nominate chuck Harris
2 - limit the council to 3 people. I would nominate chuck, nathaniel, and
pauli.
3 - add me as a permanent member of the steering council.

I would split the above into two parts: a suggestion on how to change
the governance model (first half of 1 and 2) and then some thoughts on
what to do once those changes have been made (latter half of 1 and 2, as
well as 3).

For now, since those changes are not in place yet, it's probably best
to focus on the governance model.

I would agree that one person (or a very small group) is best suited to
"getting things unstuck". And, personally, I believe it best for that
person/persons to be elected by the community (whatever we define "the
community" to be)---which is what I presume you suggested when you
mentioned nominating candidates.

Since Matthew mentioned the governance proposal we're working on, here
is a very early draft:

https://github.com/stefanv/skimage-org/blob/governance_proposal/governance.md

As I said, this is still a work-in-progress--comments are welcome.
E.g., the weighting element in the voting has to be fine tuned (but was
put in place to prevent rapid take-overs).

Essentially, we need:

- a way for community members to express disagreement without being
ousted,
- protection against individuals who want to exert disproportional
influence,
- protection against those in leadership roles who cause the project
long-term harm,
- and a way for the community to change the direction of the project if
they so wished.

Stéfan

Travis Oliphant

2015-09-22 06:08:06 UTC

Thank you for posting that draft as it is a useful comparison to borrow
from. I think Nathaniel's original document is a great start. Perhaps
some tweaks along the lines of what you and Matt have suggested could also
be useful.

I agree that my proposal is mostly about altering the governance model,
mixed with some concern about being "automatically disqualified" from a
council that can decide the future of NumPy if things don't move forward.

-Travis

and

Post by Travis Oliphant
pauli.
3 - add me as a permanent member of the steering council.

I would split the above into two parts: a suggestion on how to change
the governance model (first half of 1 and 2) and then some thoughts on
what to do once those changes have been made (latter half of 1 and 2, as
well as 3).
For now, since those changes are not in place yet, it's probably best
to focus on the governance model.
I would agree that one person (or a very small group) is best suited to
"getting things unstuck". And, personally, I believe it best for that
person/persons to be elected by the community (whatever we define "the
community" to be)---which is what I presume you suggested when you
mentioned nominating candidates.
Since Matthew mentioned the governance proposal we're working on, here
https://github.com/stefanv/skimage-org/blob/governance_proposal/governance.md
As I said, this is still a work-in-progress--comments are welcome.
E.g., the weighting element in the voting has to be fine tuned (but was
put in place to prevent rapid take-overs).
- a way for community members to express disagreement without being
ousted,
- protection against individuals who want to exert disproportional
influence,
- protection against those in leadership roles who cause the project
long-term harm,
- and a way for the community to change the direction of the project if
they so wished.
StÃ©fan
_______________________________________________
NumPy-Discussion mailing list
https://mail.scipy.org/mailman/listinfo/numpy-discussion

--
*Travis Oliphant*
*Co-founder and CEO*

@teoliphant
512-222-5440
http://www.continuum.io

Sturla Molden

2015-09-22 12:19:44 UTC

Post by Travis Oliphant
1 - define a BDFL for the council. I would nominate chuck Harris
2 - limit the council to 3 people. I would nominate chuck, nathaniel,
and pauli.
3 - add me as a permanent member of the steering council.

I have stayed out of this governance debate until now.

Personally I would prefer if you were BDFL.

Sturla

Perry Greenfield

2015-09-22 12:31:29 UTC

I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis. I don’t think anyone has invested as much personal time and resources (e.g., money) towards supporting numpy, and not just in creating it but through efforts at Continuum and Numfocus.

So much of this distrust appears based on what Continuum might do rather than what the actual record is. I agree with Travis that virtually everyone has their own interests. I don’t think non-profit institutions (academia or otherwise) are any more virtuous than for-profit companies when it comes to possible conflicts of interest. As long as everyone understands the interests involved, that shouldn’t bar anyone from participating in governance.

If anyone deserves a special seat at the table it is Travis. I’m completely convinced he has the community’s greater interests at heart (not to say that he always understand all the interests and may need input to help in that).

Perry

Sturla Molden

2015-09-22 12:51:38 UTC

Post by Perry Greenfield
I’ve also stayed out of this until now. I’m surprised and disheartened at the amount of suspicion and distrust directed towards Travis.

I have no idea where this distrust comes from. Nobody has invested so
much of their time in NumPy. Without Travis there would not even be a NumPy.

Post by Perry Greenfield
So much of this distrust appears based on what Continuum might do rather than what the actual record is.

Being CEO of Continuum should not disqualify him in any way.

Post by Perry Greenfield
I agree with Travis that virtually everyone has their own interests.

Even Guido and Linus have employers. Should we distrust Guido as Python
BDFL because some day Dropbox will be evil? It is just nonsense.

Sturla

Marten van Kerkwijk

2015-09-22 16:32:24 UTC

Hi All,

I've been reading this thread with amazement and a bit of worry. It seems
Nathaniel's proposal is clearly an improvement, even if it is not perfect.
But it is in the end for a project where, at least as seen from the
outside, the main challenge is not in governance, but rather in having only
a small group of people who understand the code well enough that they are
able to make and judge modifications. As such, this discussion doesn't seem
worthy of the effort, and even less of needless heat and irritation, of the
type that seems unlikely would have arisen if this conversation had been in
person instead of per e-mail.

Might it be an idea to accept the proposal provisionally, returning to it a
year from now with practical experience? This certainly has the benefit of
allowing to switch focus to the more pressing and fortunately also more
interesting work to be done on interfacing numpy/ndarray nicely with other
classes (i.e., __numpy_ufunc__ and/or similar, and the dtype
generalisations, which have me quite intrigued -- either might be very
interesting for the Quantity class in astropy, as well as for a
work-in-progress Variable class [which propagates uncertainties including
covariances]).
â
All the best,

Marten

--
Prof. M. H. van Kerkwijk
Dept. of Astronomy & Astroph., 50 St George St., Toronto, ON, M5S 3H4,
Canada
McLennan Labs, room 1203B, tel: +1(416)9467288, fax: +1(416)9467287
***@astro.utoronto.ca, http://www.astro.utoronto.ca/~mhvk

James E.H. Turner

2015-09-22 20:03:33 UTC

I don't think I've contributed code to NumPy itself, but as someone
involved in the scientific python ecosystem for a while, I can't see
why people would consider Continuum less of a legitimate participant
or community member than individual contributors, especially if the
person behind it has had the opportunity to control things previously
and instead passed NumPy onto the community. I'd be wary of commercial
interests dominating the agenda, but that's different from them having
a proportionate (in this case minor) say when they have something to
offer. And that's all true *even if* Travis were heavily biased to his
own commercial ends, which is not consistent with my understanding of
his wider efforts and sacrifices over most of a decade that I've been
paying attention.

Remember this is free/open source software and if enough people don't
like the committee at some point, the project can be forked as an
option of last resort. Nothing is set in stone, nor code lost.

Just saying (I probably won't reply to any criticism or corrections,
to avoid adding peripheral noise/heat to the thread).

Cheers,

James (from, but not on behalf of, a non-profit research facility).

Ryan May

2015-09-22 17:45:15 UTC