Open Source Hardware Development Method

We are working on a methodology for accelerating open source hardware development, and we are calling out open hardware practitioners to collaborate:

(note: you can re-edit the video above to help us improve it by downloading the image/voice assets and using PowToon software)

To increase our effectiveness in open hardware development, we are creating an Open Source Ecology Working Team on development workflows and standards. This is our Development Method Working Team. To do this, we are collating prior work on the topic of open hardware development and documentation by reaching out to existing open source hardware efforts. We are interested in the comprehensive process of open development, everything from documentation best practices, development workflows, and versioning strategies – to social and economic aspects of the process – such as team building, production engineering, open hardware enterprise development, and many others. If you are involved in open hardware development – please fill out this Survey of Development Workflows and Practices to provide feedback on what tools and processes you use. These results will be mede available to the rest of the open source hardware community.

Some of the qeustions we’d like to answer include:

1. What are the best platforms and practices for documentation? What development/documentation method are the most effective projects using?
2. How to best address new versions and forking in complex hardware projects while not losing any prior development effort?
3. How can open hardware projects support each other in open enterprise creation? How to fund continued development?
4. How to implement a pull request in complex, modular hardware?
5. What is a good method to capture the rationales and whys of design choices?
6. How to build and work with a team to guarantee continuity for the long term in a volunteer effort?
7. How to handle product branding and quality control when a product gains viral replication?

This is just a small sampler of unresolved questions for us.

Our goal is to facilitate open hardware projects building upon each other. One huge challenge in open hardware is essentially that every new build is a fork: tool-chains, component choice adaptations, build facilities, procedures, material sources, and other details may be different when a build is replicated by someone somewhere in the world. All these details contribute to the ‘source code’ of a piece of complex hardware – therefore – if these are different for any new build – then that build is effectively a fork. In order to achieve proper documentation so that future projects build upon all available knowledge – all such ‘source code’ must be documented – and accessible.

In the absence of documentation standards – the source code may not be ‘accessible.’ For example, if existing documentation is not well organized and it is painful to get through – someone may not read it and may start from scratch. Or, someone may not be aware that documentation exists. In both of these cases, it’s as if the documentation did not exist – and people are not building upon prior work. The documentation that exists but is not used or useful does not serve as ‘useful source’ – a term coined by Luka Mustafa of KORUZA.

Unfortunately, not building on prior work is common with open hardware. For one – much documentation is sparse. Design decisions, design rationales, and the Whys of design are rarely visible.

For example – how many hundreds of different CNC routers have been built around the world – and how many more such devices will be built – before a single, robust, marketable open source version of a high performance CNC router is created? The world of open source is plagued by such inefficiency. True, many people hack to have fun and not to meet or exceeding industry standards – but it can’t be denied that a better product is desirable and more satisfying on many levels.

Part of the inefficiency is cultural – but for those hackers of the system who are interested in real change, the overall state of open source hardware documentation and development methods is an issue that leaves much to be desired. To do better, we need adoption of standards, sharing of best practices, a clear way to find open hardware projects, quick identification of licenses, ways to fork and start new projects effectively, mechanisms to execute pull requests in complex modular hardware – and most importantly – mechanisms to facilitate adoption of these standards.

As we develop this for OSE, we think that many of our techniques and practices can be useful to the greater open hardware community.

Discussion

As the video above suggests, open source software development follows standard procedures for which tools and workflows are well-defined. Open source hardware development is a much more complicated process than software, and a coherent set of development standards is yet to be defined. For an overview of how development of open hardware takes place, your can read Building Open Source Hardware, published last December by OSE’s board member, Alicia Gibb.

With open hardware, the process is more complicated. Consider the compiler – the thing that converts the source code to a working program. In software, the compiler is another piece of software. In hardware, the ‘compiler’ includes tools, materials, the fabricator’s hand, and a worskhop space. Hardware is more complicated and expensive to manage. There are logistics, bills of materials, heavy artifacts, instructional videos, budgets – on top of the design files. The ‘source code’ in hardware includes all these assets – not only the designs, but their various descriptions – such as explainer videos and build procedures.

OSE Parameters for the Development Method

For OSE’s needs, here are some points about our approach.

1. Inviting Practitioners – We are seeking feedback from those who are actively engaged in building OSHWA-compliant open source hardware – and are steeped in the subtleties and challenges of documentation, development, its continuity, and its financial sustainability. We are looking especially for those of you who think that open hardware development has reached nowhere near its potential development velocity for accelerating innovation, and who can propose effective solutions. We are seeking feedback from entrepreneurs who are exploring innovative open source hardware business models. We are interested in practical experience more than the theoretical side. In order to join the effort, start with the Development Method Working Team application.

2. Scope – When people discuss Open Source Hardware, typically that refers to the design and documentation of blueprints. This is a subset of OSE’s interest in open hardware, and much needs to be done along this front. However, our scope is much bigger, and includes institutional experimentation of how to change frameworks and systems for facilitating the diffusion of open hardware. We are interested in enterprise development based on open hardware, and organizational innovation in order to bring about more open hardware.

3. Modularity and Granularity – The Mother of All Development Methods could take hundreds of Ph.D. theses to complete. To begin that process, we follow a granular and modular approach. Granularity means that we will seek to break down the overall methodology into small steps. Modularity refers to selecting particular aspects of overall open source hardware development- such as: Basic Documentation Process; Design/Build Process; Enterprise Creation Process – and many others. Thus, this approach will be able to accommodate many different projects and many different project states and complexities. For example, a beginner working on a small project should have access to the most basic design/documentation platform. For larger projects with more complexity which intend to scale their efforts in size and scope – such as OSE – we will have a more detailed, granular, comprehensive process that leads up to open source enterprise training.

4. Working Teams Approach – To develop different modules of the Development Method, working teams will be created. OSE will solicit contributions to the Working Team from various members of the open hardware community. When a larger number of participants joins the effort, multiple Working Teams may be created, and will organize around development of specific Development Method modules. The goal of the Team approach is to guarantee continuity and a minimum amount of effort. At first, the overall Development Method Working Group may be a single group, but once more than 6 team members are available, other groups will be created, and will coordinate directly with other Working Teams.

5. Team Governance – Merit within the Working Team is earned by contributions. Contribution to the Development Method Working Team is open to anyone. Contributions to the Survey of Workflows and Practices involve submitting proposed workflows or individual steps of development/documentation, with procedures for how these steps are carried out. This survey above is the first step. If OSE is interested in adapting a particular workflow or practice, OSE may contact the practitioner of the workflow or practice for collaboration. Contributors should include their email if they would like to be consulted further on their practices or if they would like to get involved in helping OSE develop its processes. The second step will be to discuss the submissions, and to rate these – via a discussion and up-vote mechanism. The third step will be to identify missing steps. The fourth step will be to organize the steps for coherent workflows. The fifth step may be creating templates for executing the workflows. The sixth step may be to create different workflows, depending on the audience or purpose of the workflow. For example, one template may apply to running a Design Sprint, whereas another template may be relevant to a Working Team meeting. The seventh step may involve creating a mechanism for effective version control, forking or starting of new projects. All of this should build upon OSE’s current Dozuki/Wiki development platform, shown below. For each team, the goal is for a leader to emerge such that the Team Leader will then interact with other Team Leads of the Developnent Method Working Team, while coordinating with other Working Teams.

OSE Documentation and Development Platform

Our main documentaton platform is the OSE Wiki. We use Dozuki as a graphical index to provide structure to the wiki. We also use the main website as our face to the world.

The wiki is also a sandbox which anyone can edit freely, except for critical pages such as the front page which are locked. The goal is to allow collaborative content creation, and to allow novices to document and experiment. We use Media Wiki, which is the same wiki as Wikipedia. Because of the nature of the wiki, it’s a sandbox and it is can be a mess. Don’t worry – that is ok. The key thing about a wiki is that navigation, structure, and organization can be added on top of it, either within the wiki or from an external source. Because the wiki is essentially a database, its content can be displayed in many ways. The important part of its usefulness is increasing structure, quality, and organization over time with many people.

For OSE, we are following Modular Design and OSE Specifications. Modular design means that we can track development at any scale: an entire machine ecosystem, a machine, its modules, and possibly components of modules – ad infinitum. We can treat any component of the system or the entire system as a module. This is useful because we can break down a complex product into small, bite-sized chunks or modules that a large development team can work on. When a development structure is standardized so that all the development steps are identified and clear protocols are written for each step – this allows development to occur, in principle, autonomously. When a large team is available, this in principle allows rapid, parallel development. In order to enable such parallel development, we need to know how modules fit together. For this reason, we define the interfaces between modules prior to developing the modules.

Our goal is to show that when we use this development method, drastic compression of development time is possible by highly coordinated crowd swarming. Our goal is compressing 5 person years of development time into 2 weeks of 125 person development time – or 2 weeks of 40 person-around-the-clock development time across the globe in successive time zones. We’ve been amazed to see how some people see no value in accelerated development schedules – but in our case – we look at such acceleration as the key that makes GVCS completion feasible. Historically, we have underestimated the requirements for succeeding at such a development velocity. Personally, with all the interest in the project after my 2011 TED Talk, I thought we would be done in 2 years. That didn’t happen. By now, we’ve seen enough pitfalls to think that we have figured out the secret sauce. The challenges include process completeness, clear definition of protocols, clear requirements, access to collaborative tools, access to building blocks, team building, team leadership, funding, coordination, motivation, and others. Essentially – everything can go wrong, as there are too many moving parts.

Modularity allows for parallel development, but counter-intuitively, it also crashed our documentation on the wiki. When new versions were created, it became difficult to keep track of which modules remained identical, and which ones changed. To explain, examine our documentation and development process for the GVCS:

Machines are broken into modules, and the development process takes place at the module level. The sevelopment spreadsheet reflects the numerous development steps. The spreadsheet contains a step name, a link to the protocol required to carry out that step, and a link to the product. The links all go to wiki pages, and the development spreadsheet is also a wiki page – with an embedded Google Spreadsheet.

The above allows to keep a clean graphical index of all machines, modules, and spreadsheets – while all the development and documentation is found on the wiki. Thus, the wiki is organized effectively via Dozuki.

The trouble comes when a prototype is built and tested, and a new version is begun. Because all work should be saved for learning purposes, at the point where a new version is declared, the old version is locked down and linked in the past versions of our Dozuki documentation platform:

The way the documentation platform stands today, this is required to start a new project or version:

1. New machine and all modules must be set up on Dozuki
2. All new spreadsheets must be created with new placeholder links for work product
3. All old content that remains the same must be copied into the new version. This is as simple as linking to an old version. The only caution is making sure to make a copy of a file if any change is to be made – as opposed to changing files that belong to older versions. This point is what broke the documentation on the wiki starting in 2012 – old versions were replaced with new versions, instead of copies being made for the new versions. New pages were not started in an organized fashion, but instead, old pages were updated. When there were too many fingers in the pie, this messed up the wiki. The current solution is locking down old pages when a new version is declared, so that only the new version pages are editable.

The process of setting up an entire new project thus takes 1-2 hours, plus whatever time it takes to lock down all old pages. Lockdown has to be selective, however – as effort may still be taking place on updating documentation for old versions.

1-2 hours of setup is the best we can do right now to start new projects or versions. Next steps include potentially setting all this up through templates on the wiki, so spreadsheets are eliminated and only the Dozuki placeholders have to be created.

You can see a few more words on the technical design specifications and challenges for the OSE Development Platform here. The big challenge is setting up continuity in a highly agile, networked, autonomous development process. This process consists of micro-contributions, tag-team effort, and ad-hoc effort that have such low entry barriers that they provide a highly coordinated, focused, and complex development path. We would like to break bounds of established organizational forms and funciton in a way similar to the general notions of a Distributed Autonomous Organization.

If the entry barriers are not lowered sufficiently to allow complex, advanced skill tasks to be performed by non-Spartans, then the chances of open source success in permeating the economy shall be doomed for ever:)

Other Notes on OSE Documentation

We hosted a Documentation Jam in 2013. We established a basic taxonomy tag for open source hardware (OSH) projects so that OSH projects can be found more easily on the internet. We would like to take this further by encouraging the OSH community to begin adopting this taxonomy widely so that open hardware projects can collaborate more easily.

As we move forward to build a solid developer community, we will need to develop rapid learning materials on various relevant skills, from 3D CAD, instructional video production, open source documentation, to open design. This is geared at reducing entry barriers to collaborators, and providing valuable skills. We are considering using PowToons as our platform.

The beauty of this is that if it’s worth doing, it’s worth doing badly. Explainer videos are worth doing. To address quality – we are providing the source files – so you can re-edit the video using freely-available PowToon software – and improving the script and animation. That’s the beauty of open source. Tthe source files for the video are found under the above.