Agile Isn't Enough: Here's a Better Way to Build Software that Users Want

projekt202 CTO Rob Pierry
projekt202 CTO Rob Pierry

By incorporating behavioral science with agile methods, companies can deliver software that their customers and users genuinely want and need.

In this conversation with projekt202's Chief Technology Officer, Rob Pierry explains the importance of this approach to move beyond agile and stand above the competition.

What does this idea of “beyond agile” mean?

It's a really interesting time in the software development space in particular because this idea of agile itself is gaining widespread adoption.

If you look at agile as a movement, it's been around for a lot longer than you realize; I want to say 20 years at this point, from when the manifesto was written. An entire industry has grown up around it, interested in selling new certifications and offering new training classes, and selling new certifications on top of those certifications so that you can teach your own training classes. From an education perspective, the availability of those kinds of things is a huge plus for companies looking to make these transitions, but we have to keep a clear head about the bias that's inherent in a lot of that.

They're not the ones struggling to actually deliver systems. They're interested in moving you through the class and getting you on your way. That's sort of agile itself, which fundamentally is a mechanism for how to build things. It teaches you how to account for the work you have left, how to track what you're doing, and ultimately to get things in production. What it doesn't really tell you at all is how to figure out what you should be building in the first place.

In many instances, organizations may work in an agile fashion, but while they're building in incremental ways, they're actually not releasing anything, so therefore there is no learning going on. Are you seeing the same thing?

I think that's fair to say. If you look at the original manifesto, the fundamental principle that that relates to is this idea of responding to change, preferring the ability to do that over making a plan. I think that's just acknowledging the basic reality, which is that it's not possible, things don't stay static for long enough to do iron-clad, long-term planning, and then going down with that ship even if situations have changed.

"If you start with the assumption that we are building software for people, then understanding the needs and goals and aspirations of those people just seems like the responsible thing to do."

I think there is this cultural element of, absolutely, you want to break things into smaller chunks, you want to be prepared for those shifts, but if you're not actively soliciting or investigating changes in context, then you're never going to have the input to make a change. You're prepared to change, but if you're not monitoring, if you're not getting feedback, if you're not launching things even, you're missing out on those decision points.

Agile tracks that you're building things, but are they the right things? Are they actually moving the needle with the key metrics that the business is looking for?

I think you could look at this iterative incremental idea and say, "That's the mechanism for not telling you specifically what to build, but at least giving you feedback and adjusting your path." But I think those of us who have done a sufficient number of agile projects, you'll see that that sort of feedback often ends up incremental. You're incrementally delivering and you get incremental feedback: “Man, it sure would be nice if this flow had one fewer steps in it” or “This particular interaction is a little bit awkward and I'm sure our conversion is suffering because of it.”

You rarely get feedback that says, "This thing is a total disaster and what I wanted instead was a toaster." It's not set up to solicit that kind of feedback, so that's really fundamentally where it falls down. I think you also hang out this promise of, let's get faster to market, let's get out there, let's start writing code. We actually have clients who come to us and are really disappointed that on day one of the project, some developer hasn't started coding already, like, "When are you guys going to start coding? When are you going to get going because we've got to get out there so we can hurry up and fail?" That's sort of the dominating mindset.

If you take a step back from that, I understand how people have gotten there, but it's kind of crazy. Like, everybody is all excited about failing. You even sort of see the Silicon Valley startup mentality of, "We've got to go through two or three pivots before we figure out what we're fundamentally going to do. We're going to go out there and invest a ton of time and money, even if we're doing it in an iterative, incremental fashion, before we actually land on the thing that's going to work."

Instead of “fail fast,” do you think “learn fast” is a better way to say that?

That is a little more charitable way of putting that, yeah. Fundamentally, what we think about is this notion of thinking a little bit first. Let's learn some things that's out there to be learned, right? We're not having to dig into some fundamental truth to be able to make smarter decisions.

There's a ton of metaphors you can apply; it's fundamentally, let's aim before firing. So how could we aim? Well, in the past, that aiming has been done by the product owner or the visionary CEO or something like that. Somebody has an idea and it sounds like a good idea, and the idea is probably born of experience in a space, so I don't want to discount those kinds of things, it's just that they don't scale. They don't offer predictable performance.

"When you put cross-functional teams together, if they don't understand and respect each other's crafts, it's not going to be successful."

You can have a good idea, then you might have a bunch of bad ideas. Even the canonical platonic ideal of this visionary thing, you could say Steve Jobs or something like that, Apple's had a string of failed products. They just happen to have had successful enough products that they have the bank balance to not be overly concerned by those kind of things.

So to me it seems like there's really two aspects here. There is the programmatic validation of these ideas before spending money building them, even in an iterative incremental fashion. Then, I think, related to that but perhaps even more important, is the programmatic generation of new good ideas. For us, that manifests through our work with behavioral science and generative observational research, and really adding that in as the beginning before you get to the iterative incremental churn. How are we aiming? What direction are we aiming in? What do people actually need? I'm just being a little careful. I don't want to totally discount the power of someone in the space having a good idea, but people are complicated.

The backlog is a core component of agile. Is there a better way to understand what you should focus on and what you should execute?

There's some interesting elements to that, particularly for larger companies that adopt agile and are trying to transform that way. The backlog itself is, in some respects, an artifact that represents the political power in the organization. Things get prioritized because I know it's what my boss wants or I know it's what their boss wants. They likely have sound reasoning for wanting that and there's perhaps some potential business return behind that, but sometimes there are relatively opaque political reasons for the prioritization of things.

What we suggest is buttressing that subjective judgment with objective reality. If you start with the assumption that we are building software for people, then understanding the needs and goals and aspirations of those people just seems like the responsible thing to do. So then if we decide, “Great, let's understand the people that we're building for,” and we take that as our specification for the problem we're trying to solve, then it naturally says, "Isn't there some group of practitioners who are dedicated to the understanding of people?" It turns out there is and that's fundamentally what behavioral science is.

Whether that's sociological, psychological, anthropological kind of backgrounds, that's incredibly useful. Now we have these practitioners with totally different crafts that we need to integrate into the fold of software development. That's partly where this beyond notion of beyond agile comes from. It's not just taking your development team, but it's introducing entirely new crafts into this process. On the surface, it seems pretty revolutionary, but if you go back to core agile manifesto principles, it's about cross-functional teams. We're expanding the notion of what cross-functional actually means.

What does a strong team consist of nowadays?

I would really start with this notion of behavioral science as feeding and seeding the backlog. We find that work is less iterative and incremental than you would think. It is very sequenced, you are going out, you are observing people, you are synthesizing results, you are generating new possibilities of things to build. Then perhaps you're iteratively validating those before they make it to the backlog. We do see that stripe of behavioral science, true user understanding continuing throughout the project. I think then you combine that with traditional Agile members like your developers, your quality assurance folks. Obviously, you have designers and both interaction of visual and information architecture components, and sometimes those are manifest in the same person and sometimes those are multiple people.

That is staying true to the cross-functional notion. The other important thing that often gets missed is that the business, for lack of a better noun, wants to keep the agile team at arm’s length. They do my bidding, I've prioritized the backlog, and user stories go in and production software comes out. If you really think hard about what you're trying to accomplish, the act of coding something or not coding something is a business decision. I have to be equipped to make the smartest decision about what's going to provide the most value, so that's a two-way street. That requires me as a developer to recognize that I'm also a business person.

It also requires the business person to be closer to the implementation. They are a member of the cross-functional team because they have excellence and experience in a craft, just the same as the designer or developer.

That leads to the harder thing about agile transformations, that cultural idea. It's not just, “We're going to teach everybody the rituals and, guess what, we're agile now.” No, it's a lot harder than that.

When you put cross-functional teams together, if they don't understand and respect each other's crafts, it's not going to be successful. If they don't understand that software is being written to advance some business purpose and they don't understand what that ultimate goal is, it's not going to be effective.

So communication is a key component of cross-functional teams?

It really is. I feel like the cultural aspect of things is almost more important than the particular practices and the methodology. Obviously, I'm biased, but I feel like that's something we do incredibly well. It manifests as really simple things like designers and developers eating lunch together.

That's a hard thing to get right. If you've built a relatively siloed organization with distributed responsibilities, that means first off that nobody has the whole context, so it's hard to make good decisions, but also, people don't have as much opportunity to interact. I don't understand, as a siloed developer, what motivates you as a designer. I don't understand what you care about, you don't understand what I care about, so how can we meet?

It's different points of view and differing areas of expertise. Even if we never ever talk about the project while we're having lunch, I understand you better. I know what you're motivated by and know that we're on the same team here.

I'm more willing to look at the wireframe you've produced. If I have questions, I'm going to come talk to you because you're my buddy, instead of saying, "Eh, I'm going to go download this toolkit I could use and it's close enough."

You're teaching me to see things that I haven't seen before and vice versa. Designers make decisions between two or three different options, not realizing that one of them is an order of magnitude harder to build. They're not doing that on purpose, they just don’t have the context. Those feedback loops and that communication side of things is really important, and often overlooked, particularly because a consulting company can't come in and do that for you. I can stand up your task-tracking system and I can teach you how to do stand ups, but I can't teach you how to like each other or respect each other. But the business side is just as important to that, recognizing that we're doing things for a purpose and your development teams need to be motivated.

Agile as an industry and scrum in particular does a lot of things to try to prevent that sort of counter intuitively. You can't sprint all the time, so there's some odd sort of nomenclature choices that have been made.

"To build something effectively, agile will help you, but you also want to build the right thing effectively. To do that, you have to incorporate behavioral science to truly understand who you're building for."

You as the development team don't see where things come on to the backlog, you don't understand why they're there, you have the three- or four-sentence description of the project that you're working on, but you don't know why. You probably haven't met the person that's going to be using it, so it's really hard to both care and to make good decisions.

Going out and observing people, taking photos and videos, and validating prototypes – that’s great information to bring to the development team, isn’t it?

Absolutely. It allows you to understand who you're building something for and you layer in the business context of why we're building this thing for them. You’ve got this totality of information. The kinds of things that we're able to understand by bringing in behavioral scientists just deepen that understanding. It's more the narrative that you're telling yourself about yourself when you’re using a system; what are your goals and aspirations?

We've had cases where we have done experience strategy products for people and the outcome is actually not writing any software, it's just changing the wording of the marketing message to better appeal to the target user base and it's had profound effect on those companies. The nice thing about that is there's no maintenance cost involved.

Ultimately, how would you define “beyond agile?”

I would go back to that notion of aiming before you fire. To build something effectively, agile will help you, but you also want to build the right thing effectively. To do that, you have to incorporate behavioral science to truly understand who you're building for, then the team who's implementing has to understand that they are building this software for people. The business facilitating that construction has to understand that the team is made of people, and those people need to have strong social, communication and respect bonds, so that they can effectively make the right decisions.

Follow us on LinkedIn, Twitter, Facebook and Instagram to stay current on projekt202 news.