Weeve created a short one (okay it’s two) page Overview of an SPDX Document and its fields. It is also posted on the Specifications page and in the Library.
by Jilayne Lovejoy, Legal team Co-Chair
On December 28th, 2017, version 3.0 of the SPDX License List was released. This marks the achievement of several significant milestones for the project and represents over a year’s worth of work by members of both the legal and tech teams. Below is a summary of some of the key changes that are the foundation for improved maintainability and usability of the SPDX License List into the future.
A new master format for the SPDX License List
Since its inception, the master format for the SPDX License List was a spreadsheet and text files. From these master files, other consumable formats, such as the web pages and other formats were generated. While this format was straight-forward, it was mostly maintained by one person, did not lend itself to collaboration, and thus, was not scalable for the growing magnitude and use of the SPDX License List.
Various proposals were discussed to both move away from the spreadsheet format and also improve the implementation of the matching guidelines in the master files. A format using an XML-style template for each license and tags for the various fields and matching guidelines was developed.
Converting from the old format to the new was no small task. This required an initial automated conversion followed by human checking of all 396 licenses and exceptions, some clean-up after that, and a lot of discussion to finalize the XML format along the way. The result is a new format for the SPDX License List master files as of version 3.0.
Better guidance for matching
The SPDX License List has long included matching guidelines to ensure consistent identification of licenses and use of SPDX identifiers. While the matching guidelines are human-readable/understandable, they pose challenges for full implementation in tools. This means tool-makers have been left to sort out the implementation, which can result in inconsistent matching across tools.
The new XML format captures items like bullets, copyright notices, licenses, and optional text in XML elements. This additional information allows for tools to provide better matching as per the matching guidelines.
Since release 1.2 of the SPDX specification, SPDX supported a template format which includes variable (or alternate) text and optional text. Elements such as bullets, titles and copyrights are translated into templates including the appropriate optional and variable text. With the introduction of the XML format and the significant work provided by the SPDX legal team, the number of variable or optional tags has increase from 85 in the 2.6 version of the license list to over 6,400 in the new 3.0 license list.
While this format will initially only be used internal to the SPDX legal team, that is, to generate the other consumable formats, eventually once hardened, the XML format will enable more consistent and precise implementation of the matching guidelines by license matching tools/scanners.
Clarified Identifiers for GNU Licenses
In a collaborative effort between the Free Software Foundation (FSF) and the SPDX working group to help facilitate clarity and better license identification practices, we have updated the short identifiers for the GNU family of licenses to support more precise and consistent usage.
SPDX has always had a way to identify the “this version only” and “any later version” options, for example via GPL-2.0 and GPL-2.0+ respectively. In practice, however, GPL-2.0 was not always used to mean “only version 2” as defined in the SPDX License List. It was often used by default to refer to the GPL version 2 text as drafted to include this version or any later version. The FSF was concerned about the potential confusion this could cause. Richard Stallman has posted an article explaining the background for this aspect of the GNU licenses and the root concern about copyright holders not identifying or being unclear regarding which option is intended.
By providing identifiers that are explicit as to “this version only” and “any later version”, we can be sure that SPDX users are reminded of the difference and that the right information is communicated.
As such, the next release of the SPDX License List v3.0 will reflect the changes to the GNU family of licenses following this pattern:
For each GNU license, the SPDX License List now contains a specific item and identifier for the two variations. The + operator is retained and can still be used with other licenses.
Why 3.0 and not 2.7?
A major version numbering change was used with this release to act as a clear signal to users of the license list, that this release wasn’t a business as usual. Given the significance of the changes on how licenses are represented internally, new tooling is now possible to improve the fidelity of license recognition. In addition, the short form identifier naming changes to address the FSF concerns with how the GPL licenses are represented and deprecation of the prior GNU license identifiers will impact some projects.
We want to thank everyone who made this release possible, as it was truly a team effort!
“Observers of the kernel’s commit stream or mailing lists will have seen a certain amount of traffic referring to the addition of SPDX license identifiers to kernel source files. For many, this may be their first encounter with SPDX. But the SPDX effort has been going on for some years; this article describes SPDX, along with why and how the kernel community intends to use it. ”
For more see the article by Jonathan Corbet of LWN “SPDX Identifiers in the Kernel”.
The FSFE has recommended the use of SPDX License Identifiers as part of Project Reuse.
This is a great article on SPDX and answers a lot of common questions people have: “SPDX Could Help Organizations Better Manage Their Thickets of Open Source Licenses“, by Swapnil Bhartiya.
SPDX will be meeting at the Linux Foundation 2017 Leadership Summit, being held 14-16 Februrary 2017. Information on the summit including hotels, travel and so forth is located here: http://events.linuxfoundation.org/events/open-source-leadership-summit .
This is the proposed Agenda. It is subject to change as we work the final few details out. We are also looking into getting a web share and conference phone set up for those who would like to listen in and cant attend in person. More details to follow, but it would be on best effort case. If you need an invitation code to register, please contact Kate Stewart . We hope to see everyone there!
Februrary 16th (Proposed Agenda)
9am – 10am Jilayne Lovejoy – XML working session (not on schedule). Will be summarized at the 12-12:30 session.
11am-12 pm Matt Germonprez – presenting survey results on use of SPDX. UNO has been doing interviews with SPDX users will present the results of their research on what’s working, what’s not. This is to set context for brainstorming on ways to help adoption of SPDX, discuss the issues, and possible solutions
12- 12:30 pm Jilayne Lovejoy – Review of status of XML tooling for license list and discussion of next steps needed.
1:30-2pm Mark Gisi – License expression – Review of status, and overview of problems to be tackled in 2017.
2:00-3:00 Gary O’Neall – Git plugin tooling and other tool roadmaps – Discuss scope of request to generate SPDX documents directly from GIT with a GIT plugin, which existing tools can be leveraged and plan
3:00-4:00 Yev Bronshteyn – Future File formats supported by SPDX. Yev will provide some examples of different formats based on an updated set of SPDX tools (JSON, TURTLE, …). This is to provide context for further discussions on this in 2017 and do some preliminary brainstorming on what will help the community.
4:00-5:00 Additional Topics.
Outside of meeting: Wiki team cleanup – Kate, Jack, Jilayne and Gary to do the cleanup – everyone welcome to join. Contact Jack, Kate, Gary or Jilayne
On Jan. 29, we will be moving the primary repository for the SPDX tools and License List from git.spdx.org over to github.com/spdx. this change will allow us to take better advantage of many github collaboration tools and will reduce the effort in maintaining the SPDX tools.
This will not have any impact on how the binaries are downloaded (they are already hosted on github.com/spdx).
For some time, we have been mirroring the repositories from git.spdx.org over to github.com/spdx. If you are already using github.com/spdx for read-only access the license list or tools source code, there will be no change. If you are accessing git.spdx.org, you should switch your repository access to github.com/spdx prior to January 29th. If you are a committer of code to one of the repositories on git.spdx.org, please email firstname.lastname@example.org with your github username and he will make sure you are setup for the transition.
Here are the fine detials:
If you are accessing git.spdx.org for read access to the repository:
- github.com/spdx is already mirroring the git.spdx.org repositories, so you can switch over to github.com/spdx now. It is recommended that you make the switch prior to the 29th.
- The repository names are the same and everyone should have read access.
- After January 29th, http access to git.spdx.org will redirect to github.com/spdx
- After January 29th, non-http access to git.spdx.org will no longer work (e.g. ssh)
If you are a committer to one of the git.spdx.org repositories:
- You should have already received an invitation to be a committer to the github.com/spdx – if not, please email email@example.com
- Prior to January 29th, we will continue to commit to git.spdx.org
- After January 29th, all commits will be made to github.com/spdx
- Please avoid any commits on the day of January 29
The SPDX tech team will be hosting an SPDX Tools BakeOff at LinuxCon Europe on 6 October 2016.
Particpation can be remote by phone or in person. The Bake-off (aka Plugfest) will focus on comparing SPDX Documents generated with SPDX specification 2.1 featues along with any questions people may have.
For more information and how to partipate, please read Background info for the SPDX 2.1 Bake-off in LinuxCon Europe.
If you have questions, please send email to firstname.lastname@example.org