.NET open source diversity #80
Comments
|
When .NET started, it started within the corporate enterprises, everything was under lock and key. In the past few years, with the advent of GitHub, NuGet, and some other things that have changed. Companies and developers are contributing to and using this vast amount of packages that are available. While it's still not at the level of NPM yet, I think we'll get there soon. I really believe this would continue to grow naturally as Microsoft and other organizations continue to Open Source and share there stuff. |
|
@duracellko Great question. I think what the JS community has done with NPM packages is venerable, but .NET ecosystem may not exactly need to follow suit. .NET today powers apps across a wide variety of platforms and code reusability is the key. As you mention, enthusiasts should explore existing solutions first and make libraries portable, when possible. We should invest more energy in trying to make older code .NET Standard compliant and push the envelope with new cross-platform libraries. While truly universal UI layer may be difficult, modern .NET paves way for easy sharing of libraries and this is where open source contributions can really contribute to ecosystem richness. Cheers to that :) |
|
Thanks for the question, @duracellko! Tagging @dotnet-foundation/candidates so they're aware aware this question has been asked. |
|
Very good question! This related directly back to ecosystem culture. Let's deconstruct your NPM comparison a bit. JS' culture of "move fast and break things" works well for the nascent JavaScript world, where they had to figure out a whole host of things that .NET solved a while ago (package management, build process, etc.) Many framework were popular at first, then quickly died as convergence took place (example: Bower -> NPM, Gulp/Grunt -> NPM/Webpack). The distributed nature of NPM comes with tradeoffs that I think doesn't necessarily work for the .NET ecosystem. I'm slightly off topic, so now I think I'll answer your question 🙂 Thinking as a Foundation board member, I think we'd want to think about what people really want. Going back to your other question - for those want to roll up their sleeves and contribute to open source, the Foundation could assist with connecting the people who want help to the people that need help. If there was an unmet need that a new open source contributor wanted to solve, I think another Foundation initiative could be connecting nascent contributors with people who could coach them. Put simply: New open source contributor/new project -> coaching I'd start by looking at what other similar foundations (GNOME?) have done to encourage this in the past - what worked for them, what didn't - and source more ideas from the community. |
|
I'd certainly say that the NuGet ecosystem has been greatly improved in the last few years as a result of .NET Core, both in terms of tooling and frameworks:
I do think we face one big technical hurdle in terms of NuGet packages that isn't being addressed: versioning. We're generally happy saying we support semantic versioning, but neither the language nor the runtime is really geared up for this. The diamond dependency problem is very real, and sooner or later it's going to bite us. This is something I've been concerned about for a long time, and to my shame I haven't written up the ideas I've had to potentially address it - maybe one day. But it's an area where I think the .NET Foundation should be leading the way in advocating for technical solutions to technical issues that will particularly hurt (or limit) the open source community. Having said all that, I'd like to address the last part of your question:
While I'd never particularly want to dissuade anyone from building anything new, I think it's always worth exploring what's available too. Then if you still want to start something new, at least you're doing so with your eyes open. (If you want to build another DI system, test framework or mocking framework, I'd definitely ask if you're really bringing anything new to the table.) I don't know whether it's particularly within the .NET Foundation's remit or something that might work better within Microsoft's documentation, or even as part of nuget.org - potentially with the direct support of the Foundation - but it would be great to make it easier for projects to find and evaluate options in terms of other packages. One thing we're often missing in the tech world across all platforms is guidance rather than just documentation. Creating guidance that is fair around multiple competing projects is difficult of course, but I'd rather try and fail than not try. (I would say that probably the single biggest improvement there would be nuget.org package search working rather better than it does now, but that's another matter.) |
|
@jskeet really thoughtful answer. I think you're well positioned to help solve for this problem long term. Love your vision for having the Foundation lead the way in solving for some of the more painful parts of NuGet versioning.
Couldn't have put it better myself. |
|
Lot of progress has been made recently to make NuGet much more usable, covering lot of similar use cases as npm. Maybe what is missing is a way for people to easily find and navigate projects (including similar projects). The reason is that nuget.org is still a very simple package database filled by package creator and without much meta-information. It would then become much easier to find a package that matches your needs. |
|
I think that the simplicity of "smaller amounts of well-defined packages" has some cultural drawbacks that may be a little unique to the .NET ecosystem. For a long time, the mindset of "developer as consumer" meant that devs would take the packages provided by a central source (Microsoft in this case) and would be distrustful or dismissive of packages not "blessed" by Microsoft. As many know, Microsoft has had a long and...storied history of how it approached this issue, and there are a lot of folks with strong opinions on it. For that reason, I think it's a positive thing to move towards a greater diversity of package offerings -- I think that it could normalize the idea of contributing to projects better moves away from the "consumer-first" or "consumer-only" mindsets. However, I also see an issue with going the npm route in terms of tiny, fragmented packages that build massive webs of dependencies. I think as a whole the .NET ecosystem does a pretty good job of balancing these things. In terms of new takes on existing packages vs. overcrowding the ecosystem with choices, I think we want to encourage innovation and exploration while also helping folks see that competition isn't the only way. I think there are some things that could help here:
These are just some things I've thought of so far. As a Foundation with finite time & financial resources, I think we'll need to get input on items like these and choose which of these we believe can have the greatest impact and where we can affect the most positive change. |
|
Looking at the 1700 folders and 209MB of stuff in the "node-modules" folder for the web app in one of my projects, I would say that is not something that the .NET Community should strive to emulate. While choice is great, it is sometimes really hard to find what you're looking for in the .NET open source world of GitHub + NuGet. I personally would suggest work with both of those teams to surface more features to help categorize existing solutions, and better usability for the NuGet experience inside IDEs to help find what you're looking for. Problems I see that cause people to spend time re-building existing solutions:
Although I absolutely love all of @SeanKilleen's ideas in his post (for example) and would gladly get behind them if that's what the Foundation decided, I honestly feel like putting the Foundation's weight behind advocating that other teams solve their discoverability issues would go much farther (and with less effort) than giving ourselves more work to do. |
|
My first reaction is to keep the foundation neutral on whether the package ecosystem should be broader vs deeper. Both have their pros and cons. Let's not "try to control what won't". That said, there may be interesting things if we look deeper. For example, if the trend is toward fewer packages because of a tendency for people to only trust dependencies that come from Microsoft, then that seems unhealthy, and a potential place for the foundation to show leadership. |
|
I think this is a great question and lots of interesting ideas on this thread already. What I think it needs to happen:
|
|
First of all...
I'm sitting here literally laughing out loud in the airport - and I'm certain that people are wondering why, if they only knew... "JavaScript" 🤣 I really like this question. I'm not certain if I would "wish .NET community would go that way" - it reminds me of the day that the NPM "left-pad" package broke the internet. I mean there are always pros and cons. It is a matter of encouragement. I think of the .NET ecosystem differently than I do the JavaScript ecosystem. I'm not sure there is really a right or wrong answer here either. I believe in collaboration, but it can be intimidating to attempt to contribute to an existing project. I would imagine that there are ways to improve this experience. I think you're spot on with your feelings about .NET Standard and Java and the evolution of .NET. It has come a long way and open source is making it better. I think we will continue to evolve and the .NET Foundation will help cater to the needs to this evolution. I'm excited to be a part of it. As a community we should encourage developers to collaborate but at the same time we shouldn't make it hard for them to do so. Again, another area for improvement. |
I agree, I didn't know discover.net before. And also find it complicated to navigate. |
|
I believe that many of the others have provided great feedback on this. Many of the ideas expressed I also agree with. The proper answer, I believe, to this question though is to make sure that we don't blend the role of the .NET Foundation with the roles & responsibilities of project leaders. The .NET Foundation, as described on the website is:
In furtherance of this goal, I don't believe it would be right for us to impose any arbitrary rules or suggestions as to more or less packages. However, I think the tools for collaboration and the support of individual projects might allow us a pathway to help limit the number of 'popup' libraries etc. We can do that in a number of ways.
Regardless, we want to embrace the momentum that all projects in the Foundation have and work to help to not only keep the mometum, but to grow it further. If this results in more, or less packages, I'm indifferent. I just want to see quality, supported packages. |
|
@mairaw @duracellko I’ll chime on this question from the perspective of a candidate soon, but in the meantime and at the risk of going off-topic a little, I wanted to follow-up on your good feedback regarding Discover .NET while it was fresh.
There actually is a page like this here, but clearly that needs to be better presented if multiple folks here are having trouble identifying it. I’ll try to come up with some ideas on how to better call that out (and of course, any help or suggestions are more than welcome). I’d also be curious to hear more about both of your experiences regarding the site being confusing to navigate. I don’t want to totally hijack this thread, so feel free to email/DM me or open an issue over there and I’d love to continue the conversation. |
|
Echoing others: I don't think the Foundation should recommend any particular approach to development things will flow naturally. This natural flow is largely a side effect from our tooling.
All of these define the .NET opensource ecosystem and it's culture/practices. I am not entirely sure under what mandate .NET Foundation can act here. There is an OSS guidance published by MS: https://docs.microsoft.com/en-us/dotnet/standard/library-guidance/versioning It would be great if ownership of these guidelines was transfered to the .NET Foundation and can continue to live on as community sourced best practices by our members. Further more we can create a front on the technical steering committee to encode these best practices in the the |
|
I think the .NET Foundation should not take a position on diversity vs conformity - it should promote both. Diversity encourages innovation, creativity and reveals new opportunities for community growth. Conformity encourages stability, health, longevity and helps avoid attrition and burnout. Both of these aspects need to be encouraged to provide a healthy .NET open source ecosystem. |
Great question, I hesitate to compare the two communities. The idea of adding more packages to NuGet for the purpose of having lots of packages seems silly. Diversity and choice when it comes to libraries leads to friendly competition which often leads to better libraries (American speaking, clearly). I love seeing that kind of diversity.
I would always encourage people to try and create a package as a way to learn, but not reinvent the wheel. |
The quantity of packages in a marketplace doesn't necessarily mean diversity to me - there are so many factors that tie into the size of a marketplace, from platform and language to distribution models, it's really hard to make a direct connection from the quantity of packages to diversity. Instead of focusing on the size of the marketplace or specific package distribution strategies, I would rather see the .NET Foundation be the place that developers go to for information on what's available, and push for improvements based on what is missing for developers. For example, as mentioned before in this thread, nuget's search function could really use some love. We all know this as individuals, but the .NET Foundation could play a role in representing these concerns and pushing for improvements on behalf of the individual developers. Equally, there are .NET communities out there that don't use nuget, either because they don't know about it, or because it doesn't cover their needs - maybe they're in offline environments; maybe they need private feeds and don't know how; maybe the versioning features of nuget are insufficient, maybe the IDE or environment they're in doesn't have nuget support. The .NET Foundation can be in a position to help connect these developers to the nuget team, so that the tooling can be improved for them, for instance.
There is always room for people to create new things and to contribute to existing things, and we should encourage both! The .NET Foundation can showcase what exists, sponsor the creation of guides and best practices from experts across the industry, bring together different approaches and perspectives into one place (as blog posts, docs or whatever makes sense for the content), so that developers can make more informed choices, and then go and reinvent the wheel if they really want to (who am I to judge, I've reinvented it quite a few times already! 😆) |
When it comes to software development, having a choice is always better than no choice at all. Sometimes you encounter packages that fit your needs much better than others and being able to go with them can be the different between the success or failure of your project. However, what we don't want to have is an overflowing of single-purpose libraries like we have in the Javascript world (e.g left-pad). While there's no way to prevent people from building such libraries, I believe the abundance of out of the box functionality with .NET's base class libraries will prevent that from happening.
Sometimes, people build packages with the same functionality as others for learning purposes and we shouldn't take that learning opportunity away from them. If anything, we should encourage them because it makes them more enthusiastic contributors and that's good for the community. Open source is a massive arena and there's room for everyone to play cc @duracellko |
Little bit off topic to the main question, just reacting to this. Actually I like separate elements Although that may be my personal preference. I am kind of guy who likes to dig deep. For example, when I started to work with AngularJS and some things just magically worked, I was freaking out. Although that may be the most appealing feature in JavaScript community. On the other side, when I started with WPF and XAML, I bought book from Charles Petzold about XAML and there was no XAML in first 500 pages of the book, but developing WPF in pure C#. And I loved that book and recommend it to everyone, who wants to learn XAML. |
|
In answer to the original question: Creating a novel new package doesn't preclude someone from exploring what exists out there. I personally would encourage folks to understand what's out there and consider contributing to existing projects. Especially if they're contributing a bug fix or an idea that is well aligned to the project. At the same time, if a contributor's idea is novel and travels in a different direction than another project, then by all means start a new project. From my experience, I've seen a lot of backlash in the past when people supposedly "reinvent the wheel." I think this is unhelpful because it's often shortsighted. Often, when someone reinvents the wheel, they bring something new. A lot of people gave John Resig grief for creating JQuery when Prototype.js already existed. Let's be glad he didn't listen to doubters.
I think @shana nails it. We can't focus on the numbers, but on the actual diversity of ideas implemented in code. I hope we have tons of contributors reinventing the wheel in new ways. There's no such thing as a truly original software idea that doesn't build on the work of others. Most new ideas build on the work of the past, but maybe combines a couple existing ideas in a novel manner. I'd like to see .NET OSS be seen as a source of innovation and not just ports of work done in other communities. I think this takes a lot of ideas throw out there to see what sticks. Every OSS project ultimately competes in the marketplace of attention. The place for the .NET Foundation isn't to dictate whether people create new projects or contribute to existing ones, but we can certainly provide guidance to contributors. We can provide guidance on what approach to take that will help their work make the most impact and get the attention it deserves. |
|
Excellent question, To answer, I would say .NET Foundation currently has areas where there can be improvement, I think that we still have gaps in what type of projects that as in the foundation right now. So to start I would say that the foundation needs to grow with a wider and more broad scope. Once that happens then you can take a step back and view the beauty that we would have created! All jokes aside, I think there probably is a fine line between bringing in projects that do exactly what other projects that is already part of the foundation. So it would be quite a difficult choice and would have to be made on a case-by case basis. I don't think there is a right or wrong answer here, since I think helping as many projects as possible would be amazing, but probably not realistic. |
|
When it comes to curating the projects within the .NET Foundation, there are two positions to keep in mind. First, we want the Foundation viewed as a trusted source. When someone is looking for a certain .NET project my hope is they replace the thought of "Microsoft" with a thought of ".NET Foundation". Also, it's about the process/requirements involved to donate a project to the Foundation. I believe the Foundation should never be put in a position to choose what project should or shouldn't be a part. But I also want the Foundation to be known for the support and love each project owner has, to make their contributions lasting. This balance can be accomplished in the project qualification process. More packages are not always better but too few [select] packages will not keep the Foundation top of mind in the community. I bet this is a topic that almost every OSS foundation is challenged with, every day. |
|
I can really relate to @duracellko's question, especially the background, as my daytime job is a senior Java developer and doing .NET at night. I think that it's important to fill the gaps in the ecosystem, especially when we detect that certain things from a different ecosystem work it would be a lost chance to leave this potential improvement out while "we have enough projects". We would have an "ideal", but very strongly opinionated, world when the .NET Foundation has so much power we could decide what goes and what not. That is not what I would like to see, actually it would scare me. I find it important to have room for exploration, although I don't need 30 dependency injection frameworks, this exploration does sometimes result in a whole new dimension to a project. I would really like for all .NET Foundation projects to have a vision & road map, which we probably could make together with the maintainers if they like, so we can see if the pieces of the large ecosystem puzzle fit together. This might result into some open discussions if the projects are going into the right direction, and will result in giving some advice. This might sound a bit like a contradiction, but there is a fine line we need to walk: in the end it's important to realize that the .NET Foundation is there to SUPPORT projects, not to rule them!!! |
|
Thanks for the question. My parsing of what you're asking is whether we prefer expanding the quantity of new projects to expanding the quality of existing projects (in so far as increased participation in a project leads to increased quality). My initial reaction is: The question is somewhat premised on the notion that the .NET open source community is a zero-sum pool of participants and we have to choose between encouraging them to create new projects or focus on existing ones. Personally, I reject this notion and would prefer to think of growth in our community as a reflection of how well we attract new participants and encourage existing ones to stay engaged. In that sense we can promote growth in both the number of new projects and the number of contributors to existing projects by taking steps such as making it easy to onboard to a project, fostering a welcoming and supportive attitude among participants, improving discovery of existing projects and the identification of gaps, and appealing to developers who aren't already in our community by clearly explaining the benefits of .NET and it's ecosystem. Reading deeper into the question, there's also a component of whether we should encourage both. My answer to this is an emphatic yes. People become engaged in open source for all sorts of reasons and I'm a strong believer in encouraging participation no matter the underlying motivations, whether or not other projects already exist in the same area. No matter whether someone creates a new project as a learning exercise, because they believe they have a better way of doing things, or simply because they want to make a name in the community, I want them to ship what they want to ship when they want to ship it. I don't believe it's the place of the community, the Foundation, or anyone else to direct or encourage the way in which people give freely of their time to write public code for others to use. This position often leads to the push-back that we don't want hundreds of libraries that pad a string as in those other open source ecosystems. Unfortunately, that often translates as "I don't want to take the time to figure out the best of the many available libraries to use". There's two problems with this. The first is that it assumes the effort to identify the most appropriate project is more important than the time and motivation of the developer(s) who created them. The second is that it assumes there's no good way to uncover the most appropriate project without undue effort. This is where good discovery mechanisms come into play. If we can do a good job of promoting all projects in an area and make comparing and contrasting them easy and straight-forward then it doesn't really matter how many projects are in a given area. Too much choice is only a problem if there's no good way to filter. One of the reasons I created Discover .NET was to try and make it easier to find and compare the growing number of packages in our ecosystem (which I think is awesome). I think the .NET Foundation should continue to make investments in discovery as part of an overall strategy to encourage participation. And this brings us back to the beginning: by focusing on growing our community rather than directing it, I believe we'll end up with a more vibrant ecosystem that has both quantity and quality. Edit: I intentionally didn't read the other responses before adding mine. I was both amused and not at all surprised to see much of what I said above reflected in the previous answers here. I think that's a testament to the fantastic pool of candidates we have. I'm thrilled that the Foundation will be in excellent hands no matter who is elected to the board. |
|
This might sound harsh but I do believe that the foundation shouldn't care about this. The ecosystem will go where it needs to go and we as a foundation can only facilitate. The foundation is there for the open source .NET ecosystem and if that ecosystem naturally grows into a direction then we can only embrace that. For the foundation, the question that comes closest to your original question is; should we accept small (NPM like) packages for the foundation or should we focus on the big packages? My answer would be: we should accept the .NET packages that make an impact or have the potentiation to make an impact. The board should discuss if this is the case for every project that applies. I know that I'm being a little short on my answer here but I don't want to give the impression that we can change the direction of the open source ecosystem with a grand vision here. Yes, we can all have neat ideas but in the end, that will not cut it. That said I'm happy to serve the foundation in whatever way it goes. |
and others
Question
I know that .NET is far from JavaScript world, where NPM has more packages than space shuttle has parts. Would you wish .NET community would go that way? More packages, more projects, diversity is everything.
Or would you redirect new enthusiastic contributors to rather explore and help existing solutions before starting anything new? If yes, how would you support that?
Background / More Information
When .NET Standard was about to come I said:
Java: Build once, debug everywhere.
JavaScript: One language to rule them all.
.NET: One library to rule them all.
When .NET started it was mostly compared to Java. And in my opinion one of big differences was amount of libraries/frameworks. In .NET there was single framework for desktop applications and single framework for web applications. That's what I liked about .NET.
While in Java there were several frameworks for desktop apps and several for web. And this pretty much defined open source world. And it got even more extreme in JavaScript world.
The text was updated successfully, but these errors were encountered: