In this episode of DevOps Unbound, hosts Alan Shimel and Mitch Ashley are joined by Adam Kalsey of Tricentis, Brian Dawson of theΒ Linux FoundationΒ and Caroline Wong ofΒ Cobalt to discuss why testing is considered the most frustrating bottleneck for developers and engineers. Panelists will explain the benefits of testing early on in the SDLC and how to accelerate the feedback cycle with automated and continuous testing, as well the pros and cons of testing, the value of shift-left testing and how to effectively balance quality and speed. The video is below, followed by a transcript.
DevOps Unbound is sponsored by Tricentis, the world wide leader in automated testing. DevOps Unbound airs biweekly on TechStrong TV, with all episodes available on Digital Anarchist. There is also a live monthly roundtable open to audience participation.
Alan Shimel: Hey, everyone. Iβm Alan Shimel, and youβre watching DevOps Unbound. DevOps Unbound is sponsored by our good friends at Tricentis, and it consists of a biweekly, 40, 45 minute video panel on relevant topics within the DevOps sphere, and featuring a panel of experts that rotates every other show, and really great people, and something I personally really enjoy doing.
And then, about once a month or once every six weeks, we actually have a live studio audienceβor there is no studio audiences in COVID, actually, but we have a live virtual audience that participates in our show. Todayβs show is just our panel, though, without a live audience, and Iβm really looking forward to our discussion.
Our discussion today is around testing. You know, with the pressure on developers to deliver code faster, sometimes testing is considered a road bump in the way. Testing is considered slowing down the whole process. There are a lot of developers whoβd like to push testing off that plate. Of course, no one wants to go back to testing after weβve deployed software, primarily, but there is that tension there.
Let me introduce you to our panel today; weβre gonna discuss this in depth. First of all, making his first appearance as a member of The Linux Foundation on our Tech Strong TV network is our old friend, Brian Dawson. Brian, welcome to DevOps Unbound.
Brian Dawson: Thank you, Alan. Thank you, Alan. And itβs fitting that the first panel, first webinar I do since joining The Linux Foundation is with you.
Shimel: Itβs our pleasure to have you on. Brian, whatβs your position at Linux Foundation now?
Dawson: So, at The Linux Foundation, I am functioning as the VP of Developer Relations and Ecosystem Development. So, just to help, weβre gonna help The Linux Foundation to the development community while driving more of a DevOps opinion into a growing ecosystem of solutions that support open source development.
Shimel: Fantastic. Next up is my good friend, Caroline Wong. Caroline isβoh, sheβs an anchor on many of our different shows here at MediaOps and Tech Strong. Caroline, itβs great to see you again. A little background, maybe, for people who arenβt familiar with you?
Caroline Wong: Great to be here. Thank you so much. My nameβs Caroline Wong, Iβm the Chief Strategy Officer at Cobalt. Weβre a pen test as a service company.
I actually started my DevSecOps career 16 years ago leading information security teams at eBay and Zynga. So, in both cases, we were running online operations 24/7 with millions of simultaneous users daily. And now, at Cobalt, we build security software. And, like many DevOps companies, weβve got data driven, product based teams. We value automation and failing fast, and super excited or todayβs conversation.
Shimel: Fantastic. Welcome. Great to have you, Caroline. Our third panel member is another returning DevOps Unbound veteran, and thatβs Adam Kalsey of Tricentis. Adam, welcome.
Adam Kalsey: Thanks. Thanks for having me. I lead Developer Relations for Tricentis. Iβve been building web applications since 1995, and have always had a soft spot in my heart for bad applications and broken things. And so, I love to help developers build higher quality software and do it in a more efficient, faster way.
Shimel: Absolutely, and thanks for being on with us today, Adam. Last but not least, in his Florida outfit here, visiting me in Florida, is my Co-host of DevOps Unbound, Mitch Ashley. Mitchell, welcome.
Mitch Ashley: Coming to you from the MediaOps Studio B.
Shimel: Yes.
Ashley: Great to be here. Iβm a former product, both security and software SaaS, CTO as well as a CIO, and now I run Accelerated Strategies Group, an analyst firm that focuses on those things.
Shimel: Absolutely. Hey, Mitch, thank you.
So, letβs get started. Adam, youβre with Tricentis, the sponsor of DevOps Unbound, and they’re a testing company, probably the leaders, one of the leaders worldwide in automated testing, continuous testing. Have we put too many straws on the developersβ camelβs back by making them, you know, be more responsible for testing to say we need to test more of our code earlier.
Is it a bridge too far for many developers? Is it making them resent testing? You know, whatβs the current state?
Kalsey: I think where weβre failing developers is by handing them lots of more things to do without giving them anyβwithout giving them what tools, and I donβt mean just necessarily software tools, but without giving them the things that they need to do it effectively. If we just pile things on developers and say, βOh, this is your job nowβ but we donβt give them the support, we donβt give them the training, we donβt give them the expertise, we donβt give them anything to make that betterβsure, itβs just gonna be harder on βem.
Shimel: To play devilβs advocateβoh, those poor little testers making a quarter of a million dollars a year each. Do we want βem to do testing, too? Right? Is there some of that here, Brian? Orβgo ahead, Adam?
Kalsey: No, no, Brianβplease, please, jump in.
Shimel: Well, heβs from The Linux Foundation, he now represents these folks. [Laughter]
Dawson: Well, first, not all of them make a quarter of a million dollars a year, right? But I do think what you raise is important when you bring up the investment, right? We are hiring talent with a particular expertise in a particular domain, a particular ability to solve problems and innovate.
And there is an argument to be said, Alan, that, βDo I want to spend this money, this quarter of a million dollars plus 200,000 in RSUs on having somebody write test harnesses?β Now, I thinkβyou know, that can be balanced by what Adam said. Well, first of all, yes, because we want everybody thinking about quality. But you know what, if we want a return on that investment, we wanna free the cognitive load, then you know what? We need to invest in automation, we need to invest in better collaboration and reducing friction from the process.
Shimel: Agreed, agreed. Caroline, you know, itβs been a bigβthis whole DevSecOps movement and shift left has been about shifting security testing left and getting developers more involved in security testing, right? Yet another camel on theβyet another straw on the camelβs back. Should we be doing that?
Wong: So, Iβoh, my Internet is awful. Can you hear me?
Shimel: Yes, we hear you fine.
Wong: Okay, fantastic, awesome. Iβll justβI know that thereβs post editing, so itβs okay for me to say that.
Shimel: Mm-hmm.
Wong: So, Alan, I bring a particularly unique perspective to this group in the sense that I actually oversee people at our company, I oversee Human Resources. And Iβve been with the company since we were 10 people, and today weβre 200. And one of the super interesting things that happens is, naturally, as a company grows, what people do, they specialize and then we have to figure out how to communicate more effectively.
And I think that if youβve got an engineering team and you say, βOkay, now youβre responsible for Development and youβre responsible for Ops, and your responsible for security, then thereβs got to be some level, depending on the maturity and the size of the organization and what the organizationβs trying to accomplish of specialized roles and protocols for engagement.β
And I think that when, either number one, those roles are not clear to, I think Adam was making a point earlier about security folks need to be providing guidance to engineers whereβbecause engineers are not expected to be security experts, frankly. Security experts are not expected to be engineers. And then, at the point where information is being handed off where the responsibility for taking an action, those, all of those sort of procedures, they have to be clear, otherwise they’re simply not going to happen.
Shimel: Yeah, agreed. I think they have to be clear. But even if we make them as clear as we possibly can, is it justβare we putting too much on developers? I mean, I think fundamentally, thatβs the question here, right?
Wong: Thereβsβ
Ashley: Alan, itβs not onlyβsorry, Iβm just gonna jump in. Itβs not only quantity of workβis it the right work?
Shimel: Right.
Ashley: Because I know from writing software and being part of software teams for so many years, thereβs this thing called unintended consequences or unintended use. Well, youβre not supposed to use it that way, but the person who designed it intended it to be used a certain way, but one of the important functions of tests is not just to uncover bugs and elicit those things or security issues, itβs also to push the boundaries about how you might use the software and essentially stretch it to try to find where itβs gonna break where they can. And oftentimes, thatβs a different mentality than or perspective, certainly, than the person that wrote the code.
Dawson: And if I may jump in and say maybe part of the discussion can get to generalist versus specialist, right? There is this idea, Alan, that when you talk about putting a lot in the developerβs hands, this idea that there is this prolific person called the full stack developer that can separate themselves from what they develop to look for security vulnerabilities, to test it, then ultimately deploy and run it, right? And I think we need to give thought to, are we trying to build a bunch of generalists, which doesnβt potentially enable them to do any job the best, or are we enabling specialists to better work together, share information, and crowdsource quality and crowdsource security?
And Iβm just curious if any of the other panelists have an opinion on this concept of full stack developers versus, say, needing traditional, lifelong quality experts.
Kalsey: Yeah, I like to say that we decided to create cross-functional teams and we did it by removing all of the functions, and we centralized them all in a single personβand thatβs not a cross-functional team. Thatβsβ
Shimel: Thatβs a cross-functional person.
Kalsey: Yes. Well, and you end up with the jack of all trades, master of none problem. You have to have the specialists on even a cross-functional team. You have to have, thereβs a lack of testers, thereβs a lack of UX people, thereβs a lack of security people. They have to be part of that team. You have to have people that, dedicated folks to do the hard work. A developer may be able, and a generalist may be able to do a lot of the easy things and a lot of the, βOh, yeah, weβve done this before or weβve seen this, and this is a known know, letβs go fix that.β
Itβs the hard work of figuring outβwhat donβt we know, and then having the expertise to be able to know what that hard work is, even, to go after it. And without that, of course the developers are going to be overloaded, they’re being asked to do everything and they donβt know what everything is.
Shimel: Agreed, agreed. I mean, I think, again, security is the poster child for this, right? Weβre asking developers to be security people or to make security a priority. Now, you ask most security people four or five years ago, theyβd tell you youβre smoking your socks. Developers donβt care about security. They never did. Only we care about security.
Itβs changed. Itβs changed a little bit. We do think otherβno one wants to develop insecure code. But we canβt ask them to be security peopleβor can we, Caroline?
Wong: I sort of think we have to. I think that, you know, the big, bad thing that no one wants to happen is a security breach, and a security breach cripples an engineering teamβs ability to execute on a roadmap. You know, if something big and bad happens, then entire teams have to rush and commit to fixing this problem sometimes for months, you know, time which, of course, could be invested more profitably elsewhere.
I have a quote that Iβd love to share from a colleague. Denali Lumma is the VP of Engineering over at Project Ronin, and she also makes a really good point about tech debt. She says, at a certain point, itβs too late, thereβs too much tech debt, too much complexity, the code base is too big, and it becomes impossible to do anything about it. What you see there is a business will slowly die because shipping features becomes harder and harder, critical vulnerabilities and bugs are introduced more frequently, and eventually it kills the business.
You know, I often think of software as a living organism, right? Whether you think of it as a tree or as an animal, there is this balance between making sure youβre doing sort of your fun things, your running, but youβre also resting when you need to heal. Youβre alsoβyou know, if weβre talking about a human being, youβre also sort of going to the doctor once a year.
Shimel: Agreed.
Wong: Yeah.
Shimel: Agreed. Thoughts?
Kalsey: Quality has to be a mindset, security has to be a mindset, user experience has to be a mindset on the developer time, among everybody there. The user experience person needs to understand and have a mindset of developing and of creating a secure experience and not, βWell, weβre gonna get rid of all the security because thatβs hard for people to use.β Everybody has to have that mindset, but you still need the experts and you still need the expertise to be able to do things well. Just having a mindset doesnβt make you fantastic at it. You can say, βOh, I love to get out and Iβve got an exercise mindset, so Iβm gonna go run a triathlon tomorrow.β Itβs probably not going to work.
Shimel: Probably not.
Ashley: You know, itβs interestingβwe couldβve had this conversation 20 years ago, 10 years ago.
Shimel: Absolutely.
Ashley: You know, why are we still frustrated with tests? What is it with tests that is, you know, it used to be sort of the orphan of the development process, you know, the only thing that happens later than test is documentation. Well, now, [Laughter] itβs still an issue. Why is that? Why are we so frustrated with tests today? Iβm curious what folks think.
Dawson: Iβm gonna start with one not very complex thoughtβitβs actually hard.
Shimel: It damn right is.
Ashley: A little more respectβwe need a little more respect for tests. [Laughter]
Shimel: Well, but this was like, I remember when we first launched DevOps.com, a popular theme was that, you know, DevOps would kill the testing star, right? That there would be less QA jobs because everything was gonna be automated continuously and shift left, we wouldnβt need testersβpoppycock, right? We have as many testers as we ever had, if not more. It didnβt kill off testing jobs. I think it was Adam that said weβve made testing part of the DevOps team. So, we have people on the DevOps team who are responsible for the various, you know, styles and types of testing that we do, including security.
I think whatβs happened, though, is, these DevOps teamsβwhen I first started DevOps.com, if you spoke to developers, theyβd say, βOh, this DevOps is all about the Ops.β And then youβd talk to the Ops people and theyβd say, βThis is DevOps, itβs all about Dev, itβs Dev-centric,β right? And thatβs the sign of a good compromise when they’re both pointing the finger at that.
But it has comeβat least as far as testing goes, from what I see, it has sort of tilted where, you know, as weβve shifted testing left, weβve shifted responsibility left as well, which kinda winds up closer to the Dev, or certainly to the DevOps teams.
However, though, letβs look at the alternative, and Iβm gonna ask all three of youβall four of you have been around the block once or twice. My personal feeling is, is that we do a much better job of test coverage today than we did 7 or 10 years ago, and I think DevOps and continuous automated testing is a big reason for that.
Thoughts? Anybody disagree? Show me wrong.
Wong: I will challenge you, Alan.
Shimel: Okay. I was hoping someone did.
Wong: Iβll challenge you, and the reason is becauseβso, weβre all familiar with the OWASP Top Ten.
Shimel: Very much.
Wong: There have been a handful of versions of the OWASP Top Ten since the first was produced in 2003. I believe weβre gonna see another version later this year. The current version is from 2017, and if you take the 2017 version of the OWASP Top Ten and you put it right next to the 2003 version, it is sort of mind blowing how similar they are.
Shimel: Yeah.
Wong: And what that says to me is, we actually know exactly what the problems are. We know how to find them, we know how to fix them, and we know how to prevent them, but they’re still around, and so I think thatβsβ
Shimel: They are, but are there as many of them, right?
Wong: Absolutely. [Laughter]
Shimel: You think thereβs as many cross-site scripting or buffer overflows than there was 10 or 15 years ago?
Wong: So, there are some categories that have been eliminated. You know, there are some categories that, for which we can define a secure pattern, and we can encourage folks to use that secure pattern, and for which we can actually eliminate some of these things.
But thereβs others or which we cannot. You know, I think aboutβI recently visited my sisterβs home, and they have two young children, a 2-year-old and a 4-year-old. And at night, what they do is, they run a Roomba, which is brilliant, you know? If you could spend your weekend playing at the park with your kids instead of vacuuming, of course youβd wanna do that. And the Roomba works because they sort of take the time to get all the toys off the floor and make sure everythingβs clear and once they use the Roomba, they clean it. There is this different work involved in effectively using a Roomba thatβs different from sort of manually using a vacuum.
And I do think there are these two distinct categories of testing, one category which you could say is known, repeatable, and automatable, and another for which manual testing, I believe, will always need to exist. There are types of security flaws like business logic flaws, race conditions, chained exploits that I donβt believe that in my lifetime, we will be able to figure out how to teach a computer to find these particular types of issues.
Kalsey: And itβs beyond just security testing. I mean, exploratory testing, from a tester, your automated tests are not going to find, βThis error message makes no sense in this context. I did this, I expectedβI don’t know what this error message is trying to tell me to do.β
Your automated tools arenβt going to find that. All they’re going to be able to find is, βIt didnβt work, we sent an error message. We had the things we wanted to put in the error message there.β You can test that your code is doing the right thing, but if you didnβt design it to do it right, youβre not gonna find that with an automated test.
Shimel: Fair.
Dawson: I would tend to say, for the surveys that weβve issued and Iβve read, your original statement, βHas test coverage improved significantly?ββyes, I think all indications are, we are getting better at automated testing, we are getting better at that pursuit of that thing we call test coverage.
But I think whatβs more important is, has the quality of test improved? Are we testing the right things, right, from functional testing to security testing, various shades of security testing? And Iβd say (1) yes, we have improved there by sort of the qualitative and quantitative measures that Iβve seen. But I also think, as we truly start to focus more on quality, we start to achieve more coverage, we start to implement better tests, it becomes more clear, as I think Adam and Caroline are speaking to how much more work there is to do, how much other things there are to improve testing on. Pair that with a constantly changing technological landscape, right? Weβve gotta learn new stacks, weβve gotta figure out how to test new stacks. Weβve got half of our put in in a SaaS platform, but then we deploy on prem, et cetera, et cetera, et cetera. And then you add to that that, at the same time, weβre trying to transform organizations.
So, Adam, you talked about exploratory testing; Caroline, the human element. I think one of the things that pursuit of the full stack Dev, that pursuit of the fully automated processes failed to do in a lot of orgs I speak to is, we talk quality first, but oftentimes, the DevOps transformation doesnβt involve quality engineering first, right? And I think one of the things we need to get better at is engaging with and partnering with Quality and establishing our automated and modern processes.
Kalsey: Yeah. We talk good in the testing industry a lot about shift left and that testing is moving earlier in in the process, and too many organizations have taken that as, βWell, the developers are the ones that are early in the process, so that means the developers do it.β No! Shift your damn testers left. Get them involved in the design phase. Get them involved in talking about the architecture. Get them involved in the entire thing, not shift the task to the people who are earlier in the process. Shift the whole process, please.
Dawson: Or to a script, yeah.
Ashley: Iβm gonna take a whole βnother tack, and I thought you were gonna go here, Brian, is, I think we need to shift tests down. We need full stack testing. Itβs not application testing anymore. Most of the security errors that we have are due to configuration errors. And what Caroline said was absolutely true. We still have SQL injection, cross-site scripting, and overflows may be less so. But why? Itβs because itβs not just in the app layer that those things happen, itβs in that software stack. And I know not every tester can be an expert at the stack itself, but maybe thatβs, instead of kinda dividing the DAST and SAST and the different kinds of testing that we do and just deciding, can we shift those in one direction is, letβs think about it from a depth perspective, because thatβs where a lot of those errors are coming from. Itβs not just the app.
Kalsey: Oh, yeah, and the tooling on this is also, can be super fragmented. You need one thing to test your infrastructure code, you need another thing to test your UI, you need another thing to do the securityβand they’re not, consolidating those reports and combining them and figuring out what worked and everything is hard work and thereβs nothing that does that. This is a place that Tricentis could definitely improve and improve our own tooling here.
And then, once youβve done that testing, the results are often in a silo. Okay, I found out I had a problemβhow do I fix it? How do I go see the data behind what was going on in the system to understand this and solve this? And now Iβve gotta dive into a monitoring tool or my logs or an observability tool instead of just having everything all in one place.
Shimel: So, I think thereβs aβIβm sorry, Caroline, you go.
Wong: No worries, and thank you, Alan. So, weβve talked about shift left, shift right. Mitch is kinda talking about shift up and down. I think thereβs another axis, if you will, which is sort of what Iβll call shift in and shift out. So, you can test things in and out of band, and thatβs gonna make sort of a wildly different difference in terms of how those processes run.
You know, I think 10 years ago, security people appreciated the power that came with being a gate in the process, being an end band process. You know, today, it actually makes sense, I think, for a lot of security testing to be out of band, but that removes the sort of power and authority of security people to say something like, βYou must fix this, otherwise youβre not gonna go live if itβs happening out of band.β
Shimel: Fair. So, Iβm getting a little dizzy, weβre going left, right, up, down, in, out. Butβ
Ashley: Itβs a circle. [Laughter]
Shimel: Yeah, you know, shake it all about. But here, fundamentally, though, are we just kinda dancingβno pun intendedβare we just dancing around the real issue, which is, we wanna do more pre-deploy, we wanna do more, we want better quality software, right? I think we could all wrap our heads around that idea and vote for that. We all want better quality software. Security people are at a premium. We canβt shift security people left or en masse. Testing people are also at a premium, though weβve integrated them into the DevOps teams. Developers are at a huge premium, they’re probably our most expensive resource in terms of humans.
What do we need to do to make everyoneβs job easier here, to make this function better? Is it a question weβve gotta automate more? I mean, maybe thatβs what it is, if we automate more, or is it a cultural thing, right, whereβ
Ashley: Thereβs also test-driven design, right, which is, as youβre designing it, you talk about, you put together your test strategy of, βOkay, we do it this way. Letβs talk about how we are going to test it, start to lay out the framework for how we think this can be validated.β I mean, itβs not the only thing to do, but I thinkβso, if you start with that in mind, now youβve got a path to pursue as opposed to, βHereβs a chunk of software, you folks figure out how to test it, right? We built it, you test it.β
Dawson: Yeah, and Iβd add, I think thereβs noβagain, not only is the practice of testing harder than we often like to admit, but the, you know, the path to kind of getting to where we fully wanna be, where we donβt dread testing, complex, hard path, and Iβd say itβs the DevOps trinity, you know, Alan, and we all can probably hit a few points on each.
But you have to address people and culture, as Adam brought in earlier. Thereβs no getting away from, security needs to be a thought for everybody, right? I can install all the locks I want on the front of my door. Actually, over 24 yearsβtoday is my anniversaryβmy wife has programmed me to lock the door when I come in the house without thinking, right? Sheβs kind of impressed a security culture on me.
You do have to address people and culture, process, practice, tools and technology. Automation alone will not solve it. A mindset alone will not solve it. All of that needs to come to fruition.
Shimel: Mitch, make a noteβinvite Brianβs wife to our next security panel.
Dawson: [Laughter]
Ashley: Iβve already looked up her e-mail addressβthank you, Brian, appreciate it.
Shimel: Happy anniversary, by the way, too, Brian.
Dawson: Thank you, thank you.
Kalsey: This is one of the places where I think what Caroline said is so important, that this in bound, out of boundβif youβre trying to do everything in line, of course, itβs gonna be harder. In shifting that mindset to doing things continually in the background, we already do so much stuff and process automation in the background where we donβt have people check, βHereβs a step in the process.β No, this is going along in the background, dependency watching and that sort of thing and updates.
We can be doing testing in the background as well for a lot of it, and identifying what needs to be in the foreground and what needs to be in the background. Look, Iβve seen CI/CD pipelines that take hours and hours and hours to run because the tests take six hours to execute and to build. That canβt work in any sort of a reasonable DevOps system.
Wong: I think thereβs two parts to a solution, and Iβm really, frankly, just parroting what Brian said. I think thereβs a collaboration piece between the people. You know, to an earlier part of our conversation, youβve gotta have the folks with those specialized skill sets. Ideally, youβre able to get access to those folks on demand, which historically has not been something thatβs been possible, but thatβs beginning to happen.
The second thing isβso, youβve got your people collaborating, youβve also gotta have your technology integrated. So, for whatever security tool or processors, how do those outputs actually get to the engineers? What we need is bidirectional integrations for things like Dev ticketing systems. We need things like APIs for test results. I think collaboration at the people place and integration at the technology placeβboth of which we could call processesβthen itβs just, itβs really a repeat of what [Audio skips].
Shimel: Mm-hmm. I mean, but wasnβt this one of the promises of DevOps? I mean, I got into DevOps because I thought it was a better way to do security, truth be told. But one of the promises of DevOps is, weβre gonna automate more, so we could go faster with more qualityβtwo out of three ainβt bad, right? And, you know, did we lose that somewhere? Is that no longer a goal, or have we taken that as far as it goes?
Kalsey: We keep trying to solve it with technology instead of by looking at the culture and changing how people think. And you canβt think the same way and apply new technology to it and expect the technology just to magically solve all your problems.
Β Shimel: Brian, you agree, or Caroline?
Dawson: I absolutely 100 percent agree, and Iβll agree and then Iβll just add a point. And I know Iβm being a bit nebulous about it, but thereβs a human element to this. At the end of the day, itβs humans developing software for humans, right? Whether itβs the code on your electronic control module in your car or Waze on your phone or the OS, at the end of the day, the final endpoint is a human. And you know what? At least now, and I know weβre improving, thereβs no way that we can ensureβand I donβt mean to accept security here, but to test the quality of the human experience, right?
And think about thisβat the end of the day, knowing that the language on a screen is self-descriptive and informs me what to do. Testing the fact that, or ensuring that when I submit my credit card number itβs not vulnerable, right? Whether weβre talking API testing, whether weβre talking security testing, all of this at the end of the day boils up to customer or user experience testing.
And I guess what Iβm gonna posit is that we can never entirely automate that. And where weβve hurt ourselves and possibly made a false promise is, weβre gonna create a bunch of full stack engineers, weβre gonna get rid of, away from operations functions, testing functions, and weβre gonna use TDD and automate everything from the start, Gherkin, yadda yadda yadda. And I believeβlook, that has enabled us to go faster, but our impedance mismatch, the limit weβve hit right now is where weβve failed to better connect people and enable them to collaborate and share knowledge. So, not just security, not just back end, not just data, but the entire customer experience is of quality. Dev canβt do that alone.
Shimel: No, sir.
Dawson: Iβm sorry, Alan, did I get, did Iβ
Alan Shimel: No, no, thatβs allβIβm saying no, sir, youβre 100 percent correct, but I know Caroline wanted to say something.
Wong: So, I do have a comment about automation. I think that DevOps has figured out how to do automation when it comes to defect discovery. I think weβre sort of good at that. I think what weβre still trying to figure out is automation when it comes to workflow and process so that, actually, again, to Brianβs point, the collaboration can happen automatically, rather than due to a human whoβs involved and making it happen. How do we get the right information to the right people in the easiest, most efficient way at the right time, and how do we make it super easy for them to do their jobs?
I do think that there remainsβI do think that the next evolution of automation and DevOps is gonna go from defect discovery automation, which I think weβve gotten quite good at, to workflow and process automation.
Shimel: Fair enough. Adam, thoughts?
Kalsey: I like it. I mean, identifying what youβre automating and whatβs working in that automation is important, but also identifying whatβs not being automated or what in your automation is broken. And not broken from the standpoint of, it doesnβt execute or it doesnβt run to completion, but broken in, itβs not solving the problem you wanted it to solve.
Shimel: Fair enough. Guys, we have time for maybe one more question. Iβm gonna pose it and throw it out there to you. Is it possible weβve shifted too far left or that you can shift too far left and we need a little rubber band, here, going the other way, a little pendulum swing? Or you think we just scratched the surface? Caroline, you look like you wanna say something.
Wong: I think thatβs a great question. I do think itβs possible to shift too far left, and the reason is because, while we, I think, are all familiar with the statistics and the studies that say if you fix something earlier in the cycle, itβs cheaperβthereβs stuff that exists to the right that doesnβt exist yet on the left. Software development is, you know, to sort of a little analogy that we talked about earlier, I think itβs a dance. I think itβs a dance between people. I think software comes about as a result of the behaviors and the actions and the decisions that various folks take together. And security and quality are emergent properties. They’re not Band-Aids, they’re not vitamins, they’re not painkillers. They have been, as a result of this dance. So, there are some things for which, if you try to test it too early, it doesnβt actually exist in its final form yet. If I take my 6-year-old to the doctor and then I donβt when sheβs 21, you know, might we have caught something super important as a 6-year-old that we were able to step in and address right away that wouldβve been great? Yes, but as a 21-year-oldβguess what? Sheβs gonna have different problems that were not apparent at age six.
Shimel: Spoken like a true mom and parent, right? Parents, we worry about that. Anyone else? Thoughts on that? Can we go too far left, or have we gone too far left?
Dawson: I, Alan, think it just depends. I think weβve interpreted left different ways. I think itβs a good, catchy term. Itβs an important and powerful catchy term, but as Iβve heard others discuss, we sort of overemphasized this idea of left. One, to Carolineβs point, right, we just do everything here and weβre done, good to go, right? But then this idea that we can put everything in Dev? Well, no, I thinkβI think, for what itβs meant to communicate, shifting left is an apt term.
I think in terms of ensuring that weβre doing everything we can as far left or as early in the process to mitigate issues laterβgreat. Right? But we donβt just shift security left. Right? Like Caroline said, inbound, out of bound. Actually, we need to shift security everywhere. We donβt just shift quality left, we need to shift quality everywhere.
So, I think there areβwe need to be ensuring that itβs almost zero trust software delivery, right? At every stage of the process, from concept where you should be sitting down with QA and the security team to capture the requirements to test, to deploy, to run, we need to be thinking about it, also.
Alan, Iβd say shift left is goodβshift everywhere is really what we meant to be doing, right?
Shimel: Itβs better. Great.
Kalsey: Any time we engage in sloganeering, thereβs gonna be people that pick the wrong things from the slogan. I mean, βHey, we have DevOps. That means we donβt need Ops anymore!β And thatβs not what it meant, and thereβs people that have taken the wrong cues from shift left, and instead of shifting the processes left, they shifted the tasks onto the people who were left already, and thatβs not gonna work.
Shimel: Absolutely. Hey, Mitchell, you wanna tie a bow on this?
Ashley: Iβll put a bow on this. You know, there are admittedly many, many things that I don’t know, but one of the things that I do know is that, if a developer says itβs tested, itβs ready and a QA person says itβs tested and itβs ready, five out of six dentists will choose the QA person. I will, too.
So, thereβs something there. Thereβs a reason why we donβt have DevQA or DevTest. We donβt talk about it that way, we talk about DevOps or DevSecOps. QA is extremely valid and necessary, and we canβt live without it, and I think we need to invest in helping, elevating how we test and how we think about our software applications to test in. So, those dentists will be happy.
Shimel: And Crest, the Crest test for QA. Thanks very much, Mitchell. Caroline, Brian, Adamβthanks for joining us on this episode of DevOps Unbound. Great conversation. For our audience out there, do check it out. We have another episode coming up inβwell, not next week, the week after, and then I think we have a roundtable the two weeks after that.
But for now, though, weβre gonna wrap. Many thanks again to Tricentis for sponsoring DevOps Unbound. Thank you for watching, thanks to our panel. This is Alan Shimel, weβre out.
[End of Audio]


