WIP: Add a maintenance process

[funnelfiasco]

This will undoubtedly require a lot of discussion before it's mergeable, but here's a first pass at addressing #86 and #96, which are related issues.

The gist of my initial draft is that instead of publishing the list of criteria directly to baseline.openssf.org, we publish it to a development version page. The index page has information about the project and links to versions. When we cut a new version, we copy the development.md to 'v1.2.md` or whatever we call that version.

I took the discussion in #86 as a starting point for the maintenance.md page, with one notable change: I described a process that's more continuous, with versions happening at least once a year (but more often if necessary), instead of keeping criteria sitting as open PRs for long periods.

We'll probably want to move some of the contents of README.md into the index.md file (specifically the description of the criteria) and include a link in README to the website instead, but that can be done later. We probably also want to add a "this is a work-in-progress. don't use it for evaluating your project" to the in-development output, and a version/date tag to the "published" versions.

[funnelfiasco]

IMO we can proceed with this if the general shape is right, even if we need to continue iterating on the specifics of the maintenance process for a bit.

[puerco]

I'm fine having these here for now to keep things moving, but the identifier structure, codes and overall sematics built into the string should be explained in a doc by itself.

[evankanderson]

The point here is, I think, to address my comment in Slack

So, looking at the series starting with "OSPS-AC-01" through "OSPS-AC-07", AC-01 through AC-04 are all maturity level 1, AC-05 is level 2, and AC-07 is level 3. If we find a new OSPS-AC that is maturity level 1, what number does it get?

This is probably simply due to historical accident, but it's easy for someone new to come in and see a pattern which is not intentional and cause a mess.

[funnelfiasco]

Agreed. That's in the README right now. My intent was to pull it into the index page eventually as we settle on the general shape of this. I'd also be fine putting it in this doc or separate. I think my preference in descending order is:

1. Put the explanation in index.md
2. Put the explanation in maintenance.md
3. Put the explanation in a separate file
4. Leave it in the README for people to find if they go looking for it.

[puerco]

Should this be more specific?

Suggested change

	## Identifiers
	## Retiring Identifiers for Deprecated Criteria

[evankanderson]

I think this applies to all identifiers, not just deprecated ones. Without a crystal ball, we don't know which identifiers will be deprecated in the future.

[funnelfiasco]

Evan is correct. This is about how identifiers are governed generally (although it is largely driven by discussions around retirement).

[evankanderson]

It's about identifier governance in general, because it also covers when and how identifiers can be adjusted between versions (the identifier should "mean" the same thing in the future, so that a tool or document that says Addresses OSPS-XY-46 without explicitly specifying a baseline version is not ambiguous in the future).

I don't know if it's worth explaining that philosophy in this document or not. I tend to like to see the "why" behind rules, but some people find it distracting.

[puerco]

Let's make explicit the sunsetting process:

Suggested change

	* Identifiers for retired criteria will not be used.
	* Identifiers for retired criteria MUST be marked in the OSPS baseline markup as deprecated and never reused in future revisions.

[funnelfiasco]

Makes sense. I'll update with some minor revisions to your wording.

[evankanderson]

I'm not sure I quite understand how the branching / releasing strategy will work for future versions -- will we be cutting a release branch for each baseline version? If so, should web-publish be checking out each release branch into a different sub-directory in the future?

(I'm not necessarily asking to do this now, I'm just trying to understand what's being communicated)

[evankanderson]

I think this applies to all identifiers, not just deprecated ones. Without a crystal ball, we don't know which identifiers will be deprecated in the future.

[evankanderson]

Suggested change

	* Identifiers for retired criteria will not be used.
	* Identifiers for retired criteria MUST NOT be reused. A list of these retired identifiers will be maintained in (file that contains at least OSPS-AC-06 and OSPS-BR-07)

[funnelfiasco]

I like the idea of keeping them in the yaml with a retired: <BOOLEAN> entry, which is what I think puerco's comment hinted at.

[evankanderson]

If you want to keep a single file across versions (I think you're gonna regret it, but maybe I'll learn something), you might want retired: $VERSION rather than a boolean.

[evankanderson]

Ah, you'll just check in the file as of a certain point, so then you use text-diff to find the changes over time. I think that will probably be workable with tools like blame.

[funnelfiasco]

Good point on retired: $VERSION. git blame would give the same info, but this would be friendlier.

[evankanderson]

The point here is, I think, to address my comment in Slack

So, looking at the series starting with "OSPS-AC-01" through "OSPS-AC-07", AC-01 through AC-04 are all maturity level 1, AC-05 is level 2, and AC-07 is level 3. If we find a new OSPS-AC that is maturity level 1, what number does it get?

This is probably simply due to historical accident, but it's easy for someone new to come in and see a pattern which is not intentional and cause a mess.

[funnelfiasco]

I'm not sure I quite understand how the branching / releasing strategy will work for future versions -- will we be cutting a release branch for each baseline version? If so, should web-publish be checking out each release branch into a different sub-directory in the future?

(I'm not necessarily asking to do this now, I'm just trying to understand what's being communicated)

I don't plan on using branching. I can see the appeal, but I think it makes things like publishing the web content more difficult. So the idea is that when we decide we're releasing, we'd build the current state, check in the output as docs/versions/v<version number>. The build toolchain would treat it like a static page from there on out.

I didn't include the procedure here because I wanted to keep it to things that consumers care about. What do we think of a governance/SOP.md file that has the internal operations for things like this?

[SecurityCRob]

thanks for putting this together @funnelfiasco . I'll put some thought to it and see if I come up with any new feedback beyond what the crew has given so far.