Arbitrary merge testing with Perforce
Can Perforce handle large volume of merges across branches
Perforce has proven itself as an enterprise grade source-code version management system. Must keep in mind though - Software Development is a very structured process. Source code pretty much always “moves forward” - (DEV to QA to UAT to Production). Very rarely does it move in the “other direction” (eg: UAT back to DEV); Even when it does, its mostly temporary, such as a hot-fix in a specific branch. Here, we explore Perforce’s suitability as an underlying system for Configuration-as-code. The major difference between code and config here is that config elements can (and likely will) merge in arbitrary directions.
These days, where ever you look, it feels like every one is working with git. However, Git is not suitable for our purposes because it doesn’t allow for fine grained access controls - can’t assign access to different parts of the config-tree to different people / teams. There are ways to get around this (or so I have been told), but why jump through hoops where alternate solutions exist. Then there is also the open question of supporting large repositories with git. With config elements including binary files like Excel and Word documents, repo size is bound to increase beyond what git might be comfortable with.
Subversion does not have the limitations of git, but subversion didn’t do well in arbitrary merges across branches
What are we measuring?
In a long running configuration management system, can Perforce handle arbitrary merges across branches and keep operating cleanly (with no “ghost merges” - where the source control system says there is something to merge, but actually, there is nothing to merge, and performing a merge produces revisions with “no changes”).
Write a script to run the following scenario for as long as it runs (create multiple branches and merge across them):
All the steps below are automated in a single script. Broken down here for easier understanding.
Terminology / Assumptions:
- Our setup have many environments / deployments - usually one per client
- For the purposes of this config system, an environment is a folder in the config system
- Each “client” has their UAT and PROD environments
- Hence, the terms “client” and “environment” can be used interchangeably
- DEV, UAT, PROD are environment types, and different branch roots
- DEV environments do not exist for “clients”. UAT and PROD are client specific
- Anything updated in DEV would merge down into UAT
- Anything updated in UAT, would merge down into PROD
- Simplistic big picture: All branches are kept in sync
Create a DevBase branch with a config file Create a UatBase branch - from the DEV branch Make mods to DEV, merge each to UAT Verify DEV and UAT are in sync (nothing to merge. No ghost merges) Create 5 clients. Make UAT and PROD branches for each
Add new client: Create client environment in UAT, integrate config from DEV Create client environment in PROD, integrate config from UAT Make changes to DEV. integrate all to UAT Verify DEV and UAT are "in sync" Roll out all changes to all client's UAT and PROD Verify DEV and UAT are "in sync" Make changes to DEV. integrate all to UAT Verify DEV and UAT are "in sync" Roll out all changes to even indexed client's UAT and PROD Make changes to DEV. integrate all to UAT Verify DEV and UAT are "in sync" Roll out all changes to odd indexed client's UAT and PROD Make changes to DEV. integrate all to UAT Verify DEV and UAT are "in sync" Roll out all changes to all client's UAT and PROD Verify DEV and UAT are "in sync" Make a change in one client (random indexed) PROD file Reverse integrate change to client's UAT Reverse integrate change to UAT base Reverse integrate change to DEV base Start again from the top
The loop above ran for about 7 days (about 166 hours), created just over 1200 client environments, and performed a little over 524,000 commits. Thats about 3156 commits per hour averaged over the life of the test.
At the end of the test, all files are in sync. Perforce has been able to handle all the load we could throw at it:
- No merge conflict that could not be auto-resolved
- No Ghost commits
- Successful merges in any direction
These impressive results prove that Perforce is a good choice as a backing store for a configuration system.
blog comments powered by Disqus