Asynchronous Integration for Dummies

Asynchronous integration? What!  Why on earth would I care about this?  Well if you are moving data, (big data, small data, structured/unstructured data) this kind of thing is going to come up.

Imagine that you are calling a taxi.  You dial up the taxi company and ask for a cab.  Dispatcher gets your address and says: “lem’me put ya’ hold.”   While you are on hold, he is talking over the radio trying to find an available cab to come to you. The dispatcher finds one, comes back on the line and says “cab’ll be there in 10 minutes, goodbye.”    This is Synchronous Integration.  

Why is it synchronous?  The phone line was tied up the whole time.  If there was only one phone line into taxi dispatch, a new caller would have gotten a busy signal.  

Let’s look at asynchronous integration.   Here the same call comes into the dispatcher.  She takes your location and immediately says, “Call back after 120 seconds.”  So, you call back after 120 seconds and sure enough, she says, “Cab’ll be there in 10 minutes, goodbye.“

So is async better?  The answer is nuanced and very context dependent.  When you have a lot of IO operations mixed with CPU actions async is going to help a lot.  In synchronous integration, the OS ignores the thread for scheduling till the IO operation completes. It’s not so much CPU consumption as the overhead associated with managing blocked threads (memory and inter-thread coordination).   In a nutshell, asynchronous integration is potentially more efficient, where we are basically telling the OS to perform the IO and inform the process when it’s complete.

In the end, the choice between synchronous integration and asynchronous integration is not black and white.  Asynchronous integration scales well in the case of very heavy IO operations. On the other hand, synchronous integration works well where you have a lot of CPU operations.  Sometimes asynchronous is not a practical choice. For example, the JDBC spec does not contemplate asynchronous integration. Possible? Sure, but you’d have to add a lot of complexity to the native DB drivers.   K3 supports both asynchronous and synchronous integration out of the box.  But if you are torn , or wrestling with the right approach…never hesitate to reach out to us.