My 6 Coursera (& MOOC) Study Tips

Calculus I at The Ohio State UniversitySince I’ve been really quite slack in contributing to this blog I thought I should try and re-boot.

I’ve studied a few courses now on Coursera and completed about half of them. So from my rich well of both success & failure, here are Hisso’s Tips for Actually Completing Courses Online (if you’re just mucking about, these tips won’t apply):

  1. Study one course at a time, no matter how tempting it might be to add “just one more” because it sounds really interesting. As cheap as it is to sign up, the extra courses only serve to distract and overwhelm.
  2. Use the estimated hours. If the course convener has said the course will need 8-10 hours per week they mean it. I’ve found those effort estimates pretty reliable. You might look at the video lectures and think it’s only 1-2 hours a week — but the lecturers have run this before and know that the exercises, peer-assessments and reading actually make up the bulk of the time commitment. So go with their estimate.
  3. Study what’s interesting to you, not your employer. Because presumably you’re having to do this study in your own time, so you’re going to need to rely on intrinsic motivation to finish.
  4. Keep up with the lectures and quizzes. Once you start to fall behind it can be almost impossible to catch-up, because the cumulative study time required will easily exceed your spare time available. Which leads me to…
  5. Schedule in the Study Time. Ideally this is a regular time (I schedule 7-8 am weekdays) which not only helps keep you on track but also gives you a good idea of the maximum course load you can take on. If you know you only have 5 hours a week then you know not to bother trying to complete an 8-10 hour a week course (without scheduling in additional time).
  6. Find a Friend. If you can find a friend or colleague who will do the course at the same time then your chances of completing are greatly improved.

So there you go. Guaranteed Coursera Course Completion or your money back (har har).

Continue reading

Google is not Crazy

Google is not Crazy

Last month, the New York Times ran an article which raised some questions about Google’s recent acquisitions and how they fit in with its larger strategy. Some analysts see Google’s acquisition of Nest in a positive light, however, the article noted, not everyone is convinced:

Colin Gillis of BGC Partners is more sceptical. “Do you trust Google’s management as visionaries?” he asked. The analyst questioned the Nest purchase. Making thermostats does not fit in with Google’s core advertising, he said. Neither does robotics.

In my view, this is not necessarily correct. Here’s why.

Google’s Vision vs Business Model

Google defines it’s mission as: “[...] to organize the world’s information and make it universally accessible and useful.” Their pursuit of this mission is clearly seen in Google Search, but also its academic publications search, e-mail, social, scanned books, and other services. It’s perhaps harder to see that mission in driverless cars and “thermostats” (assuming that’s what Nest’s business really is). Nevertheless, I think the mission is still there. But today I want to talk about Google’s business model. In terms as simple as its mission, what would you say Google’s Business Model is? Here’s my take:

Sell precisely targeted advertising.

The mission and the business model are not the same. That’s OK, they shouldn’t be the same. The role of the business model is to support the mission. Google’s business model is deceptively simple. Yes, they sell advertising. However for Google (and Facebook, and maybe Twitter), the point is that the advertising is precisely targeted, because of Google’s access to consumer intent through its search engine, Gmail, Google+ and other services.

Limits to Google’s Business Model

There is, however, a limitation to this business model. It is this: Most of what we do in our lives we do “offline” to Google. Most people, most of the time, buy products in physical stores. We drive cars, catch trains, visit friends & family and we do these things without necessarily letting Google know about it. If your business model was to sell precisely targeted advertising, and you realised that most consumer activity was actually happening without you knowing about it, what would you do?

Intermediation Model 2.0

What Google wants to do is intermediate our lives. A lot has been written about dis-intermediation as being the defining feature of the current change sweeping the business world. But it would be more accurately described as re-intermediation. In the past, newspapers aggregated consumer eyeballs and then sold those to advertisers. Then they got dis-intermediated (and unbundled, and out manoeuvred, and et hoc genus omne). But that hasn’t meant that you get your news directly from journalists. Rather, you get it from the new aggregators and intermediaries: Google, Facebook, and Twitter.

Similarly, Amazon disrupted the book retailing market, and with its Kindle service, is now in the position to cut out the publishers completely. But again, it hasn’t meant that you buy your books directly from authors. You buy them from the new intermediaries: Amazon, Apple, and Google. Google is in the intermediation business. By learning everything it possibly can about us, it’s able to sell very precisely targeted advertising, and effectively mediate consumer access to service providers. And it can make a lot of money doing it.[1]

Google is not “Predicting the Future.”

Let’s revisit the analyst’s reservations about Google:

“Do you trust Google’s management as visionaries?” [...] Making thermostats does not fit in with Google’s core advertising [business]. Neither does robotics.

Except, yes they do. They fit because Google’s advertising is precisely targeted based on what it knows about us. What do you think a home thermostat connected to the Internet could tell Google about the people who lived there?

What correlations might Google discover between thermostat settings and, say, disposable income? What happens when the Nest product suite branches out to gather more than just temperature data? What about noise levels? Movement? Air quality? Could a Nest sensor infer the emotional state of a household based on voice intonations? Could it infer what people are watching or listening to based on background noise? Imagine having Shazam running all the time, only it identified more than just music but also news stories, movies, TV shows… Might that be interesting to an “advertising” company?

What could a self-driving car tell Google about where people went, and how often they went there? Would the car see interesting events on its travels? What would it hear people inside talk about? Do you think it might want to talk back? What would it say? Anything here interesting to an “advertising” company?

What about robots generally? Who will the robots work for and what will they do? If we delegated service consumption to an automated system that worked for us, would a company that wants to mediate your consumption of services want to know about it? Robotics is key for Google because it offers the potential for Google to break free of the digital interface straight-jacket that is mobile and desktop computing. The robot will be the new service interface. Does anyone really doubt that owning a slice of the new service interface wouldn’t make any company salivate?

Summary

TL;DR?

  • Google is not in the “advertising” business as traditionally described. It is in the business of service intermediation. They are one of the new service intermediaries.
  • The service intermediation business requires advertising that is precisely targeted for its advertiser customers to get real value. This targeting requires large volumes of data about the target consumers. Google is missing data about our “real-world” interactions, and sensor companies like Nest can fix that for them.
  • The (automated) service interfaces of the future are not the screens and keyboards we know today. “Online services” is going to break out of the constraints of mobile and desktop computing and into the physical world via “robots” and other automated services that can manipulate their physical environment.

Google will be able to learn more, and then mediate our service acquisition, in more places and more often. So no, Mr Analyst who asked a rhetorical question that I will answer anyway: No I don’t think Google’s founders are visionary. I think they are lucky to be where they are, and they’re following up that advantage with some very canny business development.

Continue reading

We are the business

An ex-manager of mine once pointed out that we need to stop talking about “the business.” Doing so gives leverage to those claiming to represent “the business” and limits the influence of the engineering team. His was more a political observation than a call to change our mind state, but ever since then I have noticed how commonplace it is for colleagues to make vague assurances that “the business requested it,” or “the business want it like that.”

When someone uses the term “the business” they invoke shadowy high priests with absolute knowledge — but they could be referring to a clueless HIPPO, an opinionated sales or marketing exec or equally to any end user of a piece of software. The engineer should have every right to question those requirements and to request exactly whom “the business” refers to in this scenario.

At Westfield Labs we are very fortunate to work in a ‘digital’ department which combines product, design, end-users and engineering resources as equal collaborators. Most recently we have moved to truly cross-functional teams where the only direction given to us by sponsors and stakeholders is high-level: e.g. bring us more customers and more conversions through focusing on streams x and y. Sure, the product team provide the ultimate direction from a product perspective but only after close consultation and collaboration with all other relevant parties.

In this scenario it is not hard for an engineer to think of oneself as part of the business and I positively encourage my team to stop using the term ‘the business’ to refer to others as it implies that we are not an equal partner. To take it further, I actively encourage my engineers to question and understand business requirements and to shout out if they don’t make sense.

Talking recently to an engineer from a UK online retailer, he noted that his company “think of themselves as a marketing company, not a software company” and see the engineering department as a necessary expense to realising their feature requests. In a business that is so dependent on the quality of the implementation and the iterative improvements upon that implementation, it is naive to think that engineers are not equal partners.

Obviously it’s not so easy when you are working in an agency (and let’s face it, sometimes you are working from a spec and clearly not ‘equal partners’) but any enterprise that wishes to succeed in the digital age will ultimately depend on the quality of its implementation – and the feedback from those that are doing the implementing. Otherwise it will be made irrelevant by a competitor that does.

So, I entreat software developers everywhere: let’s stop talking about “the business” and start talking about: customers, stakeholders, sponsors, sales team, marketers, product team, whatever. Make it clear that we consider ourselves part of ‘the business’.

New look

I just changed the theme. Not because I didn’t like the old Chunk theme but because it didn’t credit the author on the home page and I thought that was important now that hissohathair is contributing as well. It appears that editing templates on wordpress.com is not an option so welcome truly minimal, hopefully we will have a long and fruitful relationship.

While browsing the 300+ themes available on WordPress I was reminded of the elegance with which WordPress separates content from presentation and the experiences I have had moving from a CMS toward a set of APIs that serve ‘structured data’.

At westfield.com.au we threw out our Teamsite CMS a few years ago and moved away from the idea of a meta-data driven ‘universal CMS’ toward an API that serves structured data to our Presentation Layer(s). As any OO programmer will appreciate, this allows us to store the behaviour and business rules with the data – as opposed to treating data as an anonymous blob – and we’ve never looked back. We avoided the problem of an ever-growing database schema and ‘Business Layer’ by implementing an SOA but more about that another time.

We’ve been able to provide content parity across all devices by fully embracing responsive web design and we can do this because (like the WordPress themes) there is a complete separation of data from the presentation layer. As we start to move toward providing richer editorial-style content we are really starting to reap the benefits of this approach; we store data in it’s small component parts and then tag and curate this data. This allows us to create simple editorial pages such as the recently launched Curations but as we start to discover more about our users and their tastes through the power of big data we can start to tailor our content to their individual tastes and look to serve our content algorithmically rather than through creating pages as one would have in traditional publishing.

From my perspective, well constrained CMS’s like WordPress that solve a clearly defined and well understood requirement will continue to flourish but if you’re looking to fulfil a unique requirement you’re going to have to write a lot of bespoke code so why not embrace it and give yourself that freedom and control. Leave the off-the-shelf CMS for the brochure-ware sites and the big publishing companies that have the money and patience to customise them.

 

Iterating vs Dabbling

Iterating vs Dabbling

Agilists are often urging each other to “iterate” — that is to make small, incremental changes and test each change in the market as you go.

Iterating makes sense, especially when building complex systems. In complex systems, even small changes can have unintended or unexpected effects, and by moving incrementally we give ourselves the best chance of detecting early enough to minimise the cost of correction. We’re not just talking about software bugs in the new features here. “Cost of correction” includes addressing misjudged customer requirements, pivoting to reach a better market fit, and fixing regression errors.

We can also iterate without coding and this can make a lot of sense in the very early days of product development. Deming’s “Plan, Do, Study, Act” (PDSA) cycle is helpful even in early concept development. For example, we might have a few iterations of the basic concept development, talking to potential customers and then reflecting back our understanding of their needs, just to make sure we’ve really understood, and are talking about those needs in ways the potential customers recognise.

Iterating

So Iterating is Good and we’ve all got that religion. Excellent.

What we might not have is a common understanding of what “iterate” means. It’s obviously suggestive of a cycle, but are we all using the same steps in the cycle? I’ve specifically called out Deming above because it’s a genuine iterative cycle:

  • Plan: Decide, up front, what question you’re asking and how you’re going to get the answer. If you believe you’re not asking a question then what you’re about to do is probably not an “iteration” of anything so you can stop reading now.
  • Do: Execute the Plan. Measure and gather results.
  • Study: Analyse the data gathered above. Look at what we’ve learned, and if our original question is answered.[1]
  • Act: (Action Research calls this “reflect”) Finally, we integrate the learning generated from the Study above into our product, service, processes or plans.

Looking at this in the four steps above it might be easy to miss an essential feature: the cycle repeats. That is, you are not iterating if you only do it once.

Dabbling

Apple’s App Store is littered with dabblers.

Dabbling has some shallow similarities to iterating:

  • Small things are done, then released.
  • There’s an apparent attempt to gather feedback.

And then, after that, nothing happens.

There are some particular examples I have in mind but it’s unfair to call them out because there are so many others. On the App Store the telltale sign of a dabble is that there’s been only one or two releases and they were some time ago. There’s nothing really wrong with this kind of development. Perhaps the author just wanted to learn the process, and that they’ve taken things all the way through to a release on the App Store is actually a Really Good Thing from a learning point of view.

On the other hand you can also find apps that have had one or two releases and then been abandoned because they “haven’t been profitable.” It’s easy and relatively cheap to build new apps — but it’s difficult and expensive to build new businesses.

Even large and well-funded projects can catch a touch of the dabbles. Requirements churn, stories that fiddle with colours and fonts, and funding cliff-falls are all potential warning signs that you’ve stopped being serious and have started pretending:

  • Requirements Churn: Are you implementing the same basic functionality but you’re not sure why? Are your non-functional requirements changing rapidly? Are you contemplating switching to Yet Another Framework? Take a moment to check if these changes are going through a proper iteration cycle (PDSA).
  • Polishing the Poo: In past projects I look back at the number of “0.5 point” stories and realise how useless all that was. It added up to a lot of “points” but I suspect little business value was added. There was no measurement of effects in the “Do” step, and no attempt to Study the impact. We’ll simply never know if any of that work was worth it.
  • Budget/Funding Cliff-Falls: Looking back at research projects that I’ve managed I’ve decided one mistake I made was approving “build a prototype” projects with no funding or plan locked in to check it and develop it further. We certainly intended to do those things. But without a pre-approved budget to actually do them we lost momentum, staff, and potential customer enthusiasm.

Lessons Learned

There’s nothing wrong with dabbling, but dabble consciously. Understand when you’re doing it and why. If you’re iterating, iterate consciously. Call out the specific steps in the Deming/Shewhart cycle and confirm for yourself that the results of this iteration will be used in the next.

Now get back to work.
:)

Continue reading

Should a team assign points to bugs in a sprint?

I’ve been watching a discussion on a LinkedIn Agile group about whether or not a team should assign points (or estimates generally) to a bug for inclusion in a sprint. I’m going to write about how our teams handled the question and think about it from an ‘agile values’ point of view.

The Question

I’m assuming a Scrum process here, so we’ll be talking about sprints, stories and bugs:

  • sprint is a fixed-time iteration of design, development and release.
  • story is an “increment of business value.” In other words, once this story is done, someone will be able to do something useful. It has a “level of effort” estimate, usually “points.”
  • The velocity is the number of points per sprint, usually a weighted average over the last few sprints. It’s used for forward estimates of work that can be completed.
  • bug is a defect or unintended behaviour from the system. More generally, a ‘bug’ exists when the systems behaviour does not meet a customer’s expectations.

It should be relatively uncontroversial that if a bug is found in a story during its sprint then there’s no question what happens: the story card is simply “sent back” to the developer to be fixed[1]. The question some teams seem to struggle with is what happens when a defect is found after the story has been accepted as complete, and possibly even pushed to production. Here are some options people talk about when discussing how to handle the situation in an “agile” way[2]:

  1. Take the original card, remove its “complete” status, and move it back into the backlog. Subtract its points from the velocity for that past sprint (this should lower the team’s average velocity, which is generally intended). A team may want to re-estimate the card, particularly if the bug appears particularly difficult or suggests that a deeper architectural issue or understanding of requirements is at fault. However this is generally avoided in order to maintain a consistent velocity.
  2. Leave the original card, create a new “Bug” card, and add it to the backlog. If the card is urgent, it’s often added to the current sprint immediately. This approach can have two variants which are the subject of the LinkedIn discussion:
    1. The bug card is given an estimate (usually in points). If the bug goes into the current sprint, then the team is now either over-committed and likely to fall short; or the Product Owner is being asked which of their stories is the lowest priority and can it move to the next sprint? Either way this usually means that the team’s velocity for that sprint or the next is not affected, because the same number of points are delivered.
    2.  The bug card does not have an estimate or points. Teams usually pull it into the current sprint anyway, or agree with the Product Owner if it should wait for the next sprint. In any case, the team’s velocity is likely to fall, because the bug effectively counts for “zero points”, no matter how much work it takes to fix.

Our Answer

Our teams had the same discussions and we came down on the side of not assigning points to bugs. We saw points as a useful measure of how much functionality could be delivered each sprint. A bug was generally a fix to something already delivered. Fixing a bug may deliver “value for the customer,” but really what we were doing was “making good” on an earlier promise to deliver value. If there were a lot of bugs, velocity would drop. That was intended — it surfaced an issue in the development process. The root cause could be a poor choice of software library, too much technical debt, developer capability, or an inadequate QA process. By surfacing the problem, we were on the hook for addressing those issues, and could explain to management what was going on. Looking back though, I can see that it many ways, it almost doesn’t matter. What made the difference were the values that underpinned the discussion in the first place:

  • Our highest priority was to satisfy the customer through early and continuous delivery of valuable software. Working software was our primary measure of progress.
  • We wanted sustainable development, so that the entire business team could sustain a constant pace indefinitely.

We chose a process that we felt reflected those values, but you have to consider your own situation and adapt to it. (I might also note that “individuals and interactions” are supposed to be favoured over processes…)

More Light, Less Heat

These discussions sometimes seem to get a bit heated. I’m not sure if that reflects passion or pride, but either way my advice is not to put too much emphasis on the pronouncements of experts. They’re worth listening to, but only you know what’s going on in your organisation and your team and therefore what ought to work best. In one company I worked at, the accountants became aware of “points” and started including them in their reports. They would calculate the “cost per point” and developed an obsessive concern about reducing the cost per point and increasing the points per sprint (velocity). In that kind of environment you’re sorely tempted to assign points to everything. Worse, the team starts to suffer a kind of “points inflation” where the estimates start going up, velocity starts going “up” and “cost per point” starts falling — despite the fact that things are getting worse, and not better.

This makes me want to try an experiment one day: what if developers didn’t estimate level of effort on cards at all? Instead, product owners estimated how much additional revenue stories will earn the business per annum. The estimates would be in order of magnitude, for example:

  • Fixing a CSS bug on a particular browser won’t do much, so it’s worth $10 pa.
  • Fixing a CSS bug that affects all browsers might do a bit more. Maybe it makes the site look more professional. Anyway, that might be worth $100 pa.
  • A very small amount of fraud is coming from a particular location. Block it to save $1,000 pa.
  • A small percentage of customers want to save their shopping carts as wish lists. We might estimate this would add $10,000 pa.
  • Expanding wish lists to wedding registries could bring in $100,000 pa.
  • An entire new line of business might bring in $1,000,000 pa.

An alternative would be to use ‘percentage increase’ in revenue instead.

Either way, a team’s velocity would be calculated based on business value, rather than level effort. The assumption here is that business value is roughly correlated to level of effort over the medium to long term. My hypothesis is that teams might change the way they do things:

  • Product Owners and developers are much more focused on delivering business value.
  • Large stories would have to be broken down into smaller increments of business value. A story worth “$1,000,000 pa” won’t fit and so we focus on doing something that will meet some customer needs and bring in some revenue early.
  • It would become much easier to practice simplicity: the art of maximising work not done.

No doubt there’d be unintended consequences too. Still, would like to try it. :-) Continue reading

Agile Values

When I first started out with Agile software development I went looking for an agile analogue of the “Joel Test.” I wanted some kind of check-list I could run through to ensure that what we were doing included all the “essential” elements of agile development. Although I didn’t realise it at the time, this was because I wasn’t Doing It Right, because I had misunderstood Agile.

The Part I Missed

The Agile Manifesto is surprisingly brief, and the brevity is significant. Right up front we can see that the manifesto is a statement of values:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on
the right, we value the items on the left more.

Note that the manifesto does not say that there is no value in the items on the right. What it does assert though is that the values on the left are given priority over the ones on the right.

Most software engineers are familiar with the need to make trade-offs. It’s one of the things that can make development so difficult. An example of a trade-off is the tension between user-friendliness and security (no, usability is not the same thing as user-friendliness. But I digress…). Generally speaking, the more secure we make a system, the more barriers we place around it, and the less user-friendly it becomes. This is probably what you want in a banking system, but is not so good for your mailing list sign-up page.

The important point in considering such trade-offs is that neither is more important in isolation. The two need to be considered in context and then one explicitly chosen as being more appropriate in that particular situation. We can never truly say “security is always more important than ease of use” because it’s the context, the specific application, that determines importance.

Of course in software development you’re never trading-off just two factors. There can be dozens of considerations that you need to balance. These include robustness, performance (speed), resource utilisation, simplicity, and resilience. Where there is tension between developers on the Right Way to do things, explicitly stating the priority order of these things from the business perspective can help resolve the issue.

But back to the agile values. All values are important (because that’s why we call them “values”), but we often need to prefer one over another. Consider for example the values of honesty and compassion. If a friend gave you a present that you hated, would you choose honesty, and tell them that? Or would you “prioritise” compassion, spare their feelings, and thank them? What if they did it all the time, to everyone? What if it was your partner, and you thought the poor choice reflected their level of care for you?

In any non-trivial project there are considerations, values if you will, that can conflict in this way. In the Agile Manifesto, the authors made this conflict explicit, and stated a preference for certain values over others. And one of those statements places individuals over processes and tools. That’s why there’s no check-list. When you start to become obsessed with the artefacts of agile, you stop being agile.

What I Know Now

The Agile Principles follow directly from its values. For me, the first is the most important:

“Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.”

(Emphasis added)

Often (not always, but often), conflict within a team, or between teams, is because values are not aligned. The classic example is Development and Operations. Development is all about change — because change is how they create business value. New features, better performance, better customer service. Operations meanwhile is all about stability. That’s how they create business value. That means minimising change. I was once in a meeting with Dave Thomas and the CIO of a company. Dave was there to talk about his experiences introducing Agile to large companies. During the meeting Dave made a comment about change and the CIO said: “Indeed. In fact it’s better to not make any technology changes at all.” Which is not what Dave was saying. But the CIO was filtering through a different list of prioritised values, in an environment where the worst thing ever is to make a change and have it break something.

DevOps is one response to this conflict over values, although I can’t see it helping really if the values question is not addressed up front by the two teams. This can be difficult. I once tried to get a group of managers and researchers to do pair-wise comparisons of values in order to prioritise a list. It didn’t work. They couldn’t see the point, did not accept that there are trade-offs to be made, and so did not want to choose between “Excellent Research” and “Benefit Industry.” Only later, much later, did they start to engage with that trade-off.

Lessons Learned

The “so what” of all this is that Agile is not a methodology. It’s not a process, or a set of tools, or prescribed practices (that’s what “Scrum” is). Agile is a set of values. Making those values explicit is part of the process of achieving genuine management agreement with how the team will work. And if your teams are experiencing conflict internally, or with other teams, look for a possible divergence in the prioritisation of values. Discussing which values are more important in your particular circumstance can go a long way to resolving such conflicts, even if you don’t end up agreeing on the ones that should be preferred.

PS: The Joel Test is perfectly applicable to all development styles, there doesn’t need to be an “agile equivalent.” Go ahead and use it if it will help you make useful software.