What if: integration engine

Just googled some scripting languages that snap in with C or C++, ChaiScript and Lua.

Had a discussion today with a few colleagues at the office about the cost of running the various integration engines we provide support on for our customers, they all cost a shitload of money to purchase or licence – no open source alternative available?

There entry level of purchasing or licensing is probably what’s keeping small businesses from entering the connect-everything world of today.

With the thoughts running on idle cycles in the back of my head, I decided to look for something that could fill this void. If nothing is available out there, I’m tempted to roll my own, as there is lots of small companies that could make use of some integration engine .. or any integration, between ERP, HR or any system that needs to exchange/import data from another system.

My thinking goes along the lines of a lightweight scriptable integration engine – to me that means that it should not have hundreds of megabytes or more of dependencies, it should either be self-contained (all dependencies rolled into the binary) or have soft dependencies to libraries/packages on a system. Scriptable as in dynamically load, reload and unload scripts as they are discovered, updated or removed (think flexibility), where adaptation to systems are done through scripting, message transformation are done through scripting (XML, JSON, CSV and the regular flora of enterprise-y formats), native adaptation to (S)FTP/FTP(S), SOAP, REST, SMTP and (A)MQ/JMS and MQTT should be available to provide the bread-and-butter functionality. My preferred target platform would be Debian/Ubuntu or anything Linux-based.

I could probably give a bunch of good reasons of why this would be “a good thing” but the ones that are on the top of my mind is;

  • simple deployments, both code and instances (for cloud)
  • just code
  • no build-time during development

I would dog-food this like hell, I have a mix of perl, python and php scripts running on my home server that could need a touch-up, do-over or replace if such product/package were available.

Update 2017-10-30:
After discussing this further with another set of colleagues, the “small and simple” is even more interesting as most Swedish companies have less than 10 people on staff and the really small companies are just 1-2 people. I’ve also looked, gleaned at GoLang and Rust as alternatives, I have not done any attempts of writing anything yet but are trying to find something that would cover most bases .. a Linux machine can be the old decomissioned desktop machine or the raspberry pi’s, pretty much anything with wifi or ethernet connection today are able to run Linux. As for scripting I still feel that Lua would be a good candidate.