Connections

The new EIDR API Proxy was built on the framework of Reshuffle’s integration platform for developers that provides capabilities and applications that go well beyond EIDR’s current use, according to EIDR and Reshuffle.

Reshuffle recently moved to a full open source framework that makes it possible for everyone to implement Reshuffle-based applications more quickly and more easily than with traditional custom software development, Avner Braverman, CEO and co-founder of Reshuffle, said during the Dedupe, Automation & Apps panel breakout session “Open Source Opportunities: A Reshuffle Be-In” that was part of the April 13 EIDR Annual Participant Meeting.

Reshuffle has worked with EIDR for the past year or so to develop the new API Proxy, which includes an EIDR Connector that allows for direct access to the EIDR registry without the constraints of EIDR’s current XML-based API.

Kicking off the session, Raymond Drewry, VP of EMEA operations and principal scientist at MovieLabs, noted that he’s “been involved with EIDR since the very beginning and have always believed that a lot of it is about” application programming interfaces (APIs) that allow applications to use EIDR’s IDs. “That’s just so important for its success in the long run,” he said.

EIDR is “tickled pink” about the new EIDR API Proxy, according to Richard Kroon, technical operations director at EIDR. “EIDR was built on state-of-the-art technology in 2010 and it’s still state-of-the-art for 2010,” he said with a laugh.

That’s why it was “looking for ways to expand that, make more services available [and] make them more conveniently available,” he explained. However, “going in and changing the registry, changing the core API [would] interrupt all sorts of integrated applications and cause no end of pain and anguish,” he said, adding: “In addition to the development, there is the “mission critical testing we’d have to do on top of all that to make sure it was a smooth and seamless transition.”

Instead, EIDR opted to create this new API integration based on “how people are looking at applications now,” Kroon said, noting that the API Proxy uses the JSON open standard file format, which is “certainly more popular these days for API integration than is XML and is easier to read in general and easier to use in many ways.”

According to Drewry, “building the new API was really, as APIs go, completely straightforward.”

Of course, “it’s software, so there [were] bugs” and there was some “confusion because sometimes the spec wasn’t clear [and] the documentation of EIDR itself was unclear, Drewry conceded.

“But, all in all, as industrial API development goes, it was remarkably painless so I’m really pleased,” Drewry said.

At MovieLabs, so far, Drewry has used the API Proxy “for a couple of things and will no doubt use it for other stuff,” he noted. In the first case, he used it for a presentation at a conference. “It took me only a few hours to actually get it up and running and pull out of the EIDR data only and exactly what I wanted to show,” he pointed out.

“From my point of view, the biggest win is suddenly I can integrate all sorts of data sources from public and hidden APIs that I can use,” Drewry said, adding: “As a data integration point, it’s great – much more efficient [and] very effective.”

A Developer’s Take

With Reshuffle, Outform was able to “bring together quite disparate systems to wire them in such a way that they provide good business value,” according to Anton Marsden, a software developer at that Miami, Florida-based company.

Using the platform, Outform was able to “get to prototype quickly and get very quick business feedback on a solution so, from that perspective, it’s been extremely good to work with,” he explained.

Like Lego Blocks

Reshuffle was out to “make it very easy for developers to build faster and build faster by using the systems, the APIs, the cloud services and different data sources,” according to Braverman.

“If you kind of think about that, all of these different components are basically like different building blocks,” he explained. “But some of them are square and some of them are round and some of them are triangular and what not. And how do you plug them together? Sometimes they do plug together well. Often times they don’t.”

There is a layer of Connectors that basically turns each of those blocks into a “Lego block” and now that everything is “in a unified, normalized space, it’s very easy to take all those Lego blocks and put them together and build new things,” he said.

“The Connector is basically our way to interface with a specific system, be it a data source, a data base, a cloud service… like a transcoding service from Amazon or from Google, and bring it basically into this unified environment where it’s very easy to get events out of any system and respond to them and to apply action to the different systems,” he explained further.

At Reshuffle, “developers basically build their own business logic, build their own code to bring all of those systems together and all those building blocks together and build whatever solution you want to build,” he said.

So far, Reshuffle has “open-sourced a little north of 30 different Connectors,” he said. “Some of them are for services including EIDR and some are for transcoding services from AWS and some are around storage services and some are “more oriented towards user interactions, spreadsheets, smart sheets” and “some of them are more kind of lower-level data sources and databases and processing systems.”

The API Proxy is available to the public today and anyone can step in and start using it and the EIDR Connector is published and available, according to Kroon. “If you’re doing ID resolutions as standard EIDR practice that is free and available to anyone to do a resolution,” he said. However, “if you want to do more advanced features… you have to pass through EIDR credentials to do that, which is a standard… rule we have everywhere,” he said, adding: “In the new open source world, as we start driving these things forward, the community has to come together to talk about what they want to see in the future – how they want to move forward.”

One of the things that Drewry said he especially likes about the new API Proxy is its event-based program. The EIDR Connector, meanwhile, “already adds value beyond just the EIDR API because it does provide an arguably friendlier version of the query language” than the prior API, explained, adding: “It’s not as complete. There’s some things you can’t do. But in terms of writing queries, or more importantly having software write queries, it’s significantly simpler.”

What’s Next

One of the questions that Reshuffle had when starting the project for EIDR was what should it include in the API Proxy and what should be included in the Connector, according to Braverman.

“One of the things that we haven’t done is events actually coming out of EIDR,” he said. For example, let’s say there is a new movie that has been registered and he wants to know about it and add that data to his systems, he noted. “That’s definitely something that I think would be very interesting to see going forward in the future of the EIDR Connector,” he said.

Asked where Reshuffle is going from here, he replied: “I think focusing on the open source and the community is our main goal right now, see what people are doing with it and learning more and making sure that the Connectors and the core runtime are there to support whatever they need — whether it’s new deployment models, whether it’s new connectors, whether it’s new management features. But mostly growing the community and having more people engage with the project. That’s why we open source everything.”