WARNING: THIS IS DRAFT
Transition Note
The content here is now considered historical.
Baserock Developer Experience pages describes three phases of work in Baserock
- Getting started (ie we are getting Baserock to work in the first place)
- Engineering (ie Baserock is already up - we are adding new things and changing things)
- Updating (ie we are providing ongoing bugfixes and updates to delivered/running systems)
The stories here also include Trove Maintenance, which are Baserock 'Trove SysAdmin' rather then 'Development' stories.
Agatha adds some software to a system
Agatha is assigned a task to integrate a new piece of software into one of the systems she maintains for Bertrand. She retrieves her instance of the target hardware and attaches it to her working system.
Agatha deploys a virtual machine containing a development build of the target system, into which she imports the source for the software to be added. Agatha peruses the documentation and attempts to build the software.
Agatha defines a development system including all the build-dependencies for the new software. She builds the new system and deploys it into her existing virtual machine. She restarts the virtual machine and builds the software.
Having confirmed the successful build, Agatha creates instructions for the Baserock tools to build the software for the target system. Agatha then builds this newly defined system in her working environment to validate her instructions.
Due to differences between Bertrand's vendor's environment and Agatha's development system, the software does not build successfully. The Baserock tooling places Agatha in the build environment for the failing build and provides her with a log of the failed build. Agatha adjusts the build system to dynamically detect differences in system configurations and restarts the build. Discovering that she has successfully resolved the issue, Agatha exits from the build environment and the Baserock tooling transfers her change into the source tree for the software. Agatha reviews and commits the change to the build system and attempts a fresh build of the integrated system, which is successful.
With the confidence of a successful local build, Agatha pushes her work to a branch on her local Baserock infrastructure. She deploys a development system to the target hardware and instructs the target hardware to build her newly integrated system. Agatha retrieves the build artifacts from the target system into her working system and deploys the integrated customer image to the target system for testing.
Through the review process, Agatha gets her work merged into a tree visible to Bertrand. Agatha then documents the integration and the location of the resulting tree and sends that, along with a demonstration video, to Bertrand for review.
Agatha fixes a bug for Bertrand
Bertrand has discovered a flaw while integrating a vendor's changes into his system. The documentation of the vendor changes specify a newer version of an open source library than was previously integrated. Bertrand asks Agatha to please update the upstream library and ensure that other aspects of the base system are integrated with the new version.
Agatha verifies that the required version of the library is present in the mainline Baserock infrastructure. Agatha modifies the system definition for Bertrand's system to use the newer version of the library. Agatha builds the modified system and deploys it to a virtual machine. Agatha runs Bertrand's verification suite against the new virtual machine and discovers a number of regressions.
Agatha instructs the Baserock toolset to repeat the system build and place her in the completed build tree for the first failing component. Agatha updates the failing component to work with the new version of the library. After verifying her changes against a local rebuild, Agatha exits the build environment and the Baserock toolset transfers her change to the source tree for the failing component. Agatha reviews the changes and commits to her local branch. Agatha repeats this process for each verification failure.
When complete, Agatha again builds the updated system, pushes her work to a branch on her local Baserock infrastructure. She deploys a development system to the target hardware and instructs the target hardware to build her newly integrated system. Agatha retrieves the build artifacts from the target system into her working system and deploys the integrated customer image to the target system for testing, happy to confirm that the verification suite passes in this environment.
Agatha switches to her for-upstream branch and cherry-picks the changes for the change in library version from Bertrand's branch. Agatha builds the reference systems quickly using her cached artifacts and runs the mainline verification suite against the modified reference systems. Agatha determines that there are no regressions and submits her changes to the Baserock git-based code review process.
Elise reviews Agatha's patch, confirms that it appears correct and submits it to the Baserock continuous-integration infrastructure. The CI infrastructure builds the modified reference images in a pristine environment, confirms that the verification suite passes, and commits Agatha's changes to Baserock mainline.
In parallel, through the review process, Agatha gets her work merged into a tree visible to Bertrand. Agatha then informs Bertrand that the upstream library has been updated and directs him to the tree from which he can pull the new version.
Bertrand checks out Agatha's updated system definitions and builds his system. He deploys it to his target board and runs his verification suite against the target board. Bertrand finds that there are no issues.
Bertrand rebases his integration tree onto Agatha's updated system definitions. He builds and deploys to his target board and runs his extended verification suite. Bertrand now integrates the previously failing software into the system, builds his target system, deploys it to his target board and runs his verification suite. Bertrand is delighted to discover that his integration was successful.
Bertrand releases an integrated image
Bertrand had announced that he would be releasing a new integrated image on the 1st of the month and that the submission deadline for changes was today. Bertrand rebases his integration onto Agatha's latest tree and checks the review queue for any changes submitted by his colleagues.
Bertrand finds an outstanding patch series in the review queue with positive review commentary. He merges this patch series into his integration tree.
Having completed collecting submissions, Bertrand builds his system and deploys to his target board. He runs the latest version of the extended verification suite to confirm that everything integrated properly and that the expected new verification critera now pass.
Bertrand uses the Baserock toolset to generate a manifest of the changes since the previous integrated image. He reviews the manifest to verify that all of the changes were expected and that all expected changes were present. When satisfied he pushes his integration tree to his local Baserock infrastructure and tags it with the integrated image version. Bertrand notifies the project team of the new integrated image and provides the change manifest and updated verification report.
Francis was instructed that this new integrated image would address several outstanding issues, and is expected to deploy it to all units in the field.
Francis reviews the verification report to discover which issues have been resolved. Francis then reads the change manifest and writes release notes for his users.
Francis checks out the integrated image from the local Baserock infrastructure, builds his system and deploys it to his test unit. Francis runs the verification suite to confirm the provided verification report and then performs a set of additional human-driven tests against his test unit.
Satisfied, Francis uses the Baserock toolset to generate an update file which will upgrade in-field units to the new integrated image. He re-deploys the previous integrated image to his test unit, pushes the update to it, and again performs his verification process. Receiving pass results, he pushes the new ugrade file to his local Baserock infrastructure for application to in-field units. Francis then notifies his approval groups of the availability of an update, requesting that they determine if it should be placed in the in-field update staging environment.
Christina helps Bertrand
Bertrand's employer has decided to make a cost efficiency in the design of their product. Christina's employer has won the contract to produce the revised board and board support package. When the board has been released from manufacturing, an early revision is delivered to Christina.
Christina reviews the hardware change notices and determines that the boot sequence is unchanged. She verifies that she is able to boot the new board with the bootloader from the previous hardware revision and an updated device tree.
Christina checks out the reference debug system definitions for the SoC, creates a new branch for the updated board based on those definitions, updates the device tree and builds a debug system for the updated board. She deploys this to the new board and runs the verification suite against it.
Christina reviews the verification results and discovers that the system reports that it has no audio output available. Re-reviewing the hardware change notices, Christina notes that a new audio codec has been selected for this hardware revision, Christina merges the driver for the new codec into her branch. She builds and deploys this new system to the board and re-runs the verification suite, receiving pass results.
Given the passing verification suite, Christina commits and pushes her changes to her Baserock infrastructure for review.
Christina then checks out the standard release branch for the reference board and merges her system definition changes into a new branch. She builds and deploys this updated release system to the new board and re-runs the verification suite. She receives passes from the verification suite and so she commits and pushes her release branch to her local Baserock infrastructure for review.
Once Christina's work is available in the release tree visible to Bertrand, Christina uses the Baserock tooling to generate a manifest of the changes from the SoC reference system and she sends the location of the updated system definitions and the change manifest to Bertrand.
Bertrand, having received a new board, refers to Christina's message and checks out the updated reference system definitions. Bertrand builds and deploys the reference system to his new board and runs the verification suite.
Having verified that the board is properly supported, Bertrand merges Christina's changes into his integration branch. He builds and deploys this integrated system to his new board and runs his extended verification suite which passes successfully.
Happy with this, Bertrand commits his integration and pushes it to his local Baserock infrastructure for review.
Christina gets a new toy
Christina's employer has updated their SoC and generated a new reference board. After electronic validation, a TO3 revision is made available to Christina. Christina connects the board to the debugger and verifies that the SoC is responding to the debugger as recommended by the silicon vendor.
Christina checks out the reference debug system definitions for the previous SoC reference board and creates a new branch for the new board. Christina makes a first pass at the necessary changes for the bootloader to work and builds the bootloader artifact of the debug system definition.
Christina then writes partial deployment instructions to place the bootloader onto the board for testing. She boots the board and is notified of outstanding problems through the debug interface. Using this she is able to complete her changes to the bootloader. She commits these changes, pushes the bootloader changes to her local Baserock infrastructure and submits them for review by the bootloader team.
Christina merges the changes from the core SoC support team into the kernel in her branch and builds a new debug system. She updates her deployment instructions to include placement of the kernel onto the new board. She then deploys the debug system and is able to verify that she has a working serial console onto the debug system and can access a shell.
Christina now runs the verification suite against the new board and identifies the remaining components that are not yet implemented. She merges the work from each team into her branch and builds an integrated debug system. She deploys this integration system to the new board and runs the verification suite again.
Given that the suite passes, Christina commits her changes and pushes her branch to her local Baserock infrastructure for review. She then checks out the release branch for the previous SoC reference board and creates a new branch. She merges her changes from the debug branch onto her new release branch. She builds the release system and deploys it to her new board.
Christina runs the verification suite against the release system and discovers that the storage interface does not work as expected. Christina pushes her branch for Greta to review and sends Greta the verification report.
Greta checks out Christina's branch and builds and deploys the release system to her TO3 board before running the verification suite. She reproduces the same failure as Christina experienced and is able to diagnose a fault in the storage driver and correct it. Greta pushes the fix to her team's branches for review and notifies Christina to merge the new fixed driver.
Christina merges the new fixed driver into her release system, builds the system, deploys it to her board, runs the verification suite and confirms that the issue is now resolved. Christina commits and pushes her new branch to her local Baserock infrastructure for review.
Christina checks out the mainline Baserock system definitions, adds her new deployment instructions and builds a reference system and deploys it to her board. Using the debugger, she confirms that the deployment succeeded. She commits her branch and pushes it to the Baserock git-based code-review process.
Elise reviews Christina's patch and realises that it cannot be tested by the CI infrastructure. She confirms with Christina that the necessary support patches for the kernel and bootloader are being upstreamed and submits the patch to the Baserock continuous-integration infrastructure. The CI infrastructure builds the reference images in a pristine environment, confirms that the new deployment instructions do not break the verification suite and commits Christina's changes to Baserock mainline.
Christina has her work cut out
Christina's employer has designed a new SoC, implementing a new ISA, and an accompanying reference board. Christina has been delivered an early revision of this board and been asked to prepare an initial development environment. The new ISA has received some initial support by the open source community and is widely anticipated.
Christina consults the documentation for the reference board, connects it to the JTAG debugger supplied by the silicon vendor, and verifies that her sample hardware is operational. She checks out the most recent stable branch of mainline Baserock and branches that into a new local branch for her work.
Christina updates the Baserock tooling to know about the new architecture identifier. She builds a new development system for her working environment that uses the new tooling and deploys it into her existing virtual machine. She restarts the virtual machine and merges in toolchain updates from the ISA vendor and the initial kernel support from her kernel team. She also merges the debug system definitions from her local Baserock infrastructure, and the bootloader updates from the bootloader team.
She modifies the toolchain build instructions to understand the architecture identifier used in her Baserock toolset changes and generates a cross-bootstrap system definition for the new board. She writes partial deployment instructions for the new board, and attempts to build the cross-bootstrap system.
Having encountered a build failure, the Baserock tooling places Christina in the build environment for the failing build and provides her with a log of the failed build. Christina adjusts the build system to cross-build to the target ISA and restarts the build. Confirming that the cross-build succeeds, she exits from the build environment and the Baserock tooling transfers her change into the source tree for the software. Christina reviews and commits the change to the software and attempts a fresh build of the cross-bootstrap system, which is successful.
Christina deploys the successfully built cross-bootstrap system to her new board, boots the new board into the cross-bootstrap environment and verifies that the board attempts to make a native build of the remaining components.
The C library build fails, having not properly detected the new ISA when built natively. The cross-bootstrap orchestration exits in the build directory for the C library, and displays the final section of the build log, leaving Christina in an interactive shell.
Christina adjusts the C library build system and performs a local compile to verify her fix. Satisfied by a successful local build, Christina transfers the affected files to her working environment's source tree of the C library and builds a new cross-bootstrap system, deploys it and monitors it as it again attempts to make a native build of the remaining components.
This time a later library build fails for similar reasons. The cross-bootstrap orchestration exits in the build directory for that library, displays the final section of the build log, and leaves Christina in an interactive shell in which she fixes the issue and verifies that the library now builds. Christina once more transfers the affected files to her working environment and builds a new cross-bootstrap system from the cached artifacts and the updated library source. She deploys it and monitors it as it again attempts to make, and this time succeeds with, a native build of the remaining components.
Christina writes a system definition for a development system for the new ISA and commits it to her branch along with the changes she made during the debugging of the cross-bootstrap process. She pushes it to her local Baserock infrastructure on her branch. Christina then checks out her branch on the target board and attempts to build the development system for the new board.
Due to idiosyncrasies in the new ISA, the system does not build successfully. The Baserock tooling places Christina in the build environment for the failing build and provides her with a log of the failed build. Christina adjusts the software to work with the new ISA and restarts the build. Discovering that she has successfully resolved the issue, Christina exits from the build environment and the Baserock tooling transfers her change into the source tree for the software. Christina reviews and commits the change to the software and attempts a fresh build of the development system, which is successful. Christina then pushes her change to her local Baserock infrastructure on her branch.
Christina uses the Baserock tooling to transfer the built artifacts for her new development system to her updated working environment and adjusts her partial deployment instructions to transfer the full debug development environment to her new board. She deploys the natively built system to her board and boots it. Christina then runs the verification suite against her new board and gathers the report of all missing functionality.
Christina submits her branch for review, and having received favourable responses she sends the details of her branch and the verification report to her colleagues in the various driver and enablement teams so that they can reproduce her development system and begin their work in earnest.
Greta checks out Christina's branch, builds and deploys the cross-bootstrap system, allows it to complete its native build, and begins to debug the storage issues identified by the verification report shared with her by Christina.
Having received approval from her management for public release, Christina prepares a fresh branch of the current development trunk of mainline Baserock and cherry-picks her tooling changes into it. She builds a development system and deploys a fresh VM to her working environment. Within that development environment she checks out her work and verifies that the updated mainline development system is able to build the cross-bootstrap system for the new board. Christina modifies the mainline baserock verification suite to include verification of building a cross-bootstrap system for the new ISA, she confirms this does not break the verification suite, commits her change and submits it to the Baserock git-based review process.
Elise reviews Christina's patch and realises that it cannot be fully tested by the CI infrastructure. She confirms with Christina that the necessary support patches for the kernel and bootloader are being upstreamed and submits the patch to the Baserock continuous-integration infrastructure. The CI infrastructure builds the cross-bootstrap systems in a pristine environment, confirms that the new deployment instructions do not break the verification suite and commits Christina's changes to Baserock mainline.
Donald discovers Baserock
Donald's team has been slow in getting updates for their appliance platforms. Donald's management has asked Donald to find a means to faster turnaround from user requests to feature deployment.
Donald searches for platform building tools and finds the Baserock website. Being technically minded, Donald focusses on the wiki and tries the 'Getting Started' instructions. With a very short sequence of commands, Donald is able to have a working Baserock development environment in a virtual machine on his laptop and access system configuration for this environment using Xcode.
Continuing to follow the instructions, Donald builds a reference cloud appliance and prepares to deploy it on his development cloud. By creating a deployment definition using Xcode, in less than an hour Donald is able to successfully deploy the reference appliance to his development cloud.
Encouraged by his success, Donald creates a system definition for one of his organisation's appliances, using components already present in Baserock. He easily builds this appliance and deploys it to his development cloud.
Excited by the ease with which he is able to use Baserock to deploy updated systems, Donald peruses the available whitepapers and is able to crib from them to generate a justification he believes his management will approve, allowing him to standardise his development process around Baserock.
Bertrand upgrades his Trove
Bertrand's team has a Trove. They have been using it for 2 years. It's fully configured and it contains all the software and system definitions that the team mantains. It's an old version and it doesn't have the new features of the latest version. He wants to upgrade it to the newest version published in git.baserock.org.
The trove is running in a VM, and he has access to it and to its disk image. He decides to copy the disk image to his laptopt to perform a test of the upgrade in a local VM before upgrade their production Trove
Bertrand has a Baserock development system already on his laptop.
Bertrand builds the Trove system from the latest Baserock release and he deploys it as an upgrade against his local Trove, specifying the IP address of it and copying his ssh-key to it.
After the upgrade, when the new system is booting, Bertrand sees a kernel panic through the virtual machine manager, so he decides to rollback to the old system/kernel. Bertrand reboots the system and chooses the old version from the bootloader menu, leaving the local Trove running and ready to try another upgrade after he knows what the problem was.
After testing the Trove system on a different VM he realises there is a problem in the kernel of the latest Baserock release. He applies a fix to his system branch and tests it on his testing VM. The system boots. After check that it boots, he deploys the patched version of the Trove as an upgrade to his local Trove.
His local Trove boots and works fine after the upgrade. Bertrand submits the fix as a patch review to Baserock.
Now Bertrand is sure the new version of the Trove will work on their production Trove, he upgrades it on the same way he upgraded his local Trove before.
With the new version of Trove running on their production trove, he removes the old version of Trove because is not longer needed.