DevOpsβ ultimate aim is to create efficiencies in the software delivery process. But it doesnβt always work out that way. Sometimes DevOps initiatives lead to mistakesβsomething we lovingly call a βDevOoops.β CloudBees recently polled some colleagues and came up with five examples of a DevOoops in action. Following is a discussion submitted by solution architect Will Refvem about siloed departments.
Scenario:
One of the biggest challenges in DevOps is SCM. Early in my career, being given access to Git repositories was a bit like handing a belt-fed machine gun to a drunken toddler. My biggest DevOoops moment came when I was trying to master the basics of Git in a project that used Git submodulesβdonβt, just donβtβand had a series of bash scripts that implemented a primitive CI/CD workflow by running in the environments we were deploying to (seriously). I canβt tell you how many merge conflicts and detached HEADs I encountered, or how I got out of them. Fortunately, we were using the fork-PR model, so I did no real damage.
Software configuration management (SCM) is key not only for versioning, but also for providing a shared view of code and other assets, from development through to production. Giving proper consideration to your SCM structure, branching and merging model, access controls, etc., will support collaboration, reliability and traceabilityβall key for DevOps success.
In the 2019 DevOps and Jenkins Community Report, 66 percent of organizations respondents reported the use of modern solutions SCMs, specifically GIT-based solutions GitHub or BitBucket. But the older SCM solutions arenβt going away completelyβ45 percent of all organizations still use pre-GIT, pre-DVCS (distributed version control systems) such as Perforce, Subversion and CVS. While itβs possible to apply DevOps practices using these legacy solutions, it is important (and more challenging) to design a branching/merging model and surrounding workflow, which supports the key aspects of DevOps such as peer review, collaboration and short-lived branches.
Modern SCMs such as GitHub have features that inherently support collaboration, including the game-changing Pull Request, making it much easier to implement processes that best support CD and DevOps.
Poorly designed CD workflows or inadequate CD systems create friction multiple ways. If the interactions are too cumbersome, users will not commit code to the pipeline frequently enough. If itβs difficult to access the pipeline and create visibility into the process, it can lead to unresolved conflicts.
You need to pick the right tool. If you donβt have the option to get the modern tool, you need to take the time to design a legacy tool.
Itβs also important to consider two other issues in relation to this DevOoops example: branching and acquiring the appropriate level of DevOps skills.
- Branching: A proper branching and merging model is required to ensure you are feeding the appropriate inputs into your DevOps pipeline, such that you are able to continuously integrate, test and ultimately deploy changes. A simple guide here is encapsulated in the KISS (keep it simple, stupid) principle. If you need a full-wall whiteboard or a spreadsheet fit for an accountant to list your active branches, you need to rethink things. Get as close to the ideal of mainline trunk development as possible, with short-lived feature branches, minimal application versions, etc.
- Skills: As organizations seek to implement DevOps, most likely new tools and practices will be implemented simultaneously. Itβs important to understand and accommodate the fact that there will be bumps and bruises as you learn to use these new tools and practices. For instance, shifting from centralized version control such as Apache Subversion (SVN) to distributed version control such as Git is a paradigm shift that can be difficult to fully grasp and result in destructive mistakes, such as deletion of nodes from the change history, overwriting changes or misconfiguration access control. Consider hiring for these new skills and/or account for time and training.
Takeaways
- A proper branching and merging model are required to ensure that you are feeding the appropriate inputs into your DevOps pipeline to let you continuously integrate, test and ultimately deploy changes.
- If you are using a pre-Git legacy SCM solution, develop a branching and merging strategy within the legacy solution to achieve appropriate velocity in your DevOps model.
- Consider that you will need to develop new skills to adopt new SCM tools and practice and account for this in your plan.




