In recent years, APIs have quickly become part of the default application architecture. More and more sites, apps and businesses have started using a quickly growing number of third-party APIs to integrate valuable functionality into their product or gain access to a bigger, connected audience. And with cool mashups being built all over the web, their potential has even begun to enter the minds of non-techies (mostly managers): “Isn’t there an API for that?”
I’m a big fan of (well-formed) API’s, as there are quite a bit of upsides when using APIs. Development cycles can be shortened as APIs deliver ‘free functionality’, where developers don’t need to reinvent the wheel every time (and benefit from active developer communities). This enables them to really focus on the core functionality of their product. As an added bonus, a lot of stuff can be automized through clever use of APIs and seeing that most of them are hosted in elastic clouds, you never even have to worry about scaling. Right?
Though the benefits might be plentiful, recent changes in the rules of the road of the Twitter API, updated pricing on the google maps API and the rise of a platform like app.net gave a little insight into the risks and (hidden) costs of third-party API development. This has made a lot of developers think twice when considering the use of a third-party API. And rightly so! Personally, I’ve been involved in quite a few projects where the use of an API seemed like ‘the right thing to do’ early on in the development stage, only to reveal the true costs later on…
(Hidden) costs to consider when using 3rd party APIs
1) Contract management & scaling costs
Connecting to an API usually requires agreement to the terms of service of the provider. Or in short: you’re entering a contract. Now this usually isn’t a big deal at first, but when your service begins to grow, so does the amount of contracts. Also, scaling up might not be limited technically, but it might mean you’re moving upward in the pricing schemes of the APIs you are using. This is also the moment when vendor lock-in comes into play, as prices will usually go up as soon as you’ve scaled up just enough to truly reap the benefits of connected API.
2) Whose data is it anyway?
Even if you connect to an API for free, there’s always a price to be paid. Or, as wise man once said: “If you are not paying for it, you are the product.” Now, in a lot of situations, this doesn’t have to be a problem. However, you should always take this fact into consideration.
3) Not all APIs are created equal: custom development & integration pains
While things are improving, there’s still A LOT of difference in APIs. In fact, no two APIs are alike. This can make for longer and more painful integration than planned. However awesome an API might be designed, you should always account for customization, as you’ll almost never be able to integrate it with your own product ‘out of the box’. This complexity quickly grows when you want to use multiple APIs. Or as Matt Mullen says, APIs are (unfortunately) not Lego:
Not only are APIs certainly not like Lego, they are not equal. Talk to a developer and you’ll find out pretty quickly that they range from the well-formed and functional to the fiendishly complex and arcane. Then ask about the documentation. Then probably buy them a beer to recover from having to relive personal nightmares.
The chances are that every piece of software you purchase is going to have to work with a range of others that you already own. Unlike Lego, they won’t snap together simply. You’ll have to tease them in place via an integration.
4) Easy != necessary. Beware of feature creep!
The most popular APIs have a wealth of documentation and examples, trying to make integration easier for developers. While this may or may not work as well as expected, this brings up a more philosophical question: should you add functionality, just because you can? Or just because the marginal costs of connecting an extra API are low? For most developers, the answer is a straightforward ‘of course not!’. However, I’ve been part of numerous brainstorm-sessions, product meetings, etc, where feature creep was fueled by the apparent ease and availability of APIs, leading to bloated products, lack of focus with the development team and even failure of projects alltoghether.
My take? Don’t connect an API, just because you can. Ever. I’ve seen way too many products with totally tweaked out viralesque mechanisms made possible by the APIs of Facebook, Twitter, etc. Unfortunately, most of the time those products lack the required focus on their core value proposition, making it highly shareable, yet very mediocre products. Only if you have serious strategic reasons to use a third-party API, it should be eligble to come up for discussion with the development team (and restrain marketeers, as they tend to be overly optimistic about both the work required and the benefits of any integration!). And even then, be prepared to drop it.
5) Support: what if the 3rd party API fails?
Building a product or service using an external API is one thing. Maintaining that service is an entirely different thing alltogether, even more so if you don’t have ANY control over parts of your infrastructure. Have you planned for contingencies if your Paypal-API-connection goes out for an undisclosed amount of time? What if you hit the rate limit of your geocoding-API? Take this into consideration and prepare for the worst. Something WILL go down when your service hits the prime-time.
6) Roadmap alignment
This might be the catch-all of hidden costs, because at the end of the day you should realize that the API you are connecting to is not your product and ultimately, the business goals it supports are not your business goals. You have NO CONTROL over this, and you can only hope that the API roadmap stays in aligment with yours (or at least stays backwards compatible). The recent developments relating to the Twitter API show a change in direction can endanger whole companies built entirely on top of external APIs, but there’s a host of other (less strategic) problems when roadmaps start running out of sync. I have had entire teams wasting valuable time updating their code to integrate new API-versions that didn’t turn out to be backwards compatible. Or had costly updates of a whole suite of apps being forced upon us due to a simple API function update. Even if you prepare for these updates in terms of budget, the API-provider will still own your timetable and resource planning when it decides to change its API. This issue has come into play at one point or another in pretty much every single project I have been involved in and you should be acutely aware that every API connection creates a business dependency.
When (and how) to use 3rd-party APIs
Though this post might appear to lead to the thought of not using external APIs all-together, I still highly recommend using them, as I believe the benefits of (properly) implemented APIs easily outweigh the costs. I do however advise you to think beyond the first integration (the easy part) and plan for long term, strategic value (the hard part). Integrating APIs in your product or service needs to be seen as a long term activity and evaluated accordingly. In general, make sure:
- your team can stay focused and doesn’t get distracted by continuously changing API requirements.
- to plan beyond the launch, considering scaling issues & other contingencies. Always evaluate different solutions and API providers! Make sure you have the right reasons to pick one option over the other.
- you have alternative options and backup plans in place, preferably well before you need them!
- you fully understand both the short- and long-term implications of integrating a third-party API.
- dedicate resources and budgets strictly for API implementations. This also helps you get an better understanding of the true costs of APIs.
Even when these points are taken into consideration, I would strongly urge everyone to think carefully about the inherent risks of creating products and services that depend too heavily on third-party APIs. If you don’t, it could come back to bite you…
