Comparison between luis.ai vs api.ai vs wit.ai?
Artificial Intelligencewit.aiAzure Language-UnderstandingDialogflow EsArtificial Intelligence Problem Overview
Does anyone know the specific differences and features among the three, Or if one has more features/more flexible to use as a developer?
Artificial Intelligence Solutions
Solution 1 - Artificial Intelligence
wit.ai vs Dialogflow vs luis.ai
╔══════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════╗
║ wit.ai vs api.ai(Dialogflow) vs luis.ai ║
╠══════╦════════════════════════════════════╦═════════════════════════════════════════════╦════════════════════════════════════╣
║ S.No ║ Wit.ai ║ Api.ai(Dialogflow) ║ Luis.ai ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 1 ║ Wit.ai API is completely free ║ Api.ai Has a paid enterprise option ║ LUIS is in beta and free to use ║
║ ║ with no limitations on ║ which allows for this to be run on a ║ 10K transactions per month ║
║ ║ request rates. ║ private cloud internally and more ║ and up to 5 requests per second ║
║ ║ ║ from their services team., After google ║ for each account. ║
║ ║ ║ acquisition they are providing free ║ ║
║ ║ ║ services by integrating google cloud ║ ║
║ ║ ║ services. ║ ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 2 ║ Provides a nice combination ║ Speech to Text and Text to Speech ║ LUIS uses machine learning ║
║ ║ of both voice recognition and ║ capabilities, along with machine ║ based methods to analyze ║
║ ║ machine learning for developers. ║ learning. ║ sentences. To perform machine ║
║ ║ ║ ║ learning, LUIS breaks an ║
║ ║ ║ ║ utterance into "tokens". ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 3 ║ Has two main elements to it ║ Support of Intents, Entities, actions ║ Supports Intents, Entities ║
║ ║ that you set up within your ║ and one key focus area is its “Domains”. ║ and actions. ║
║ ║ app – intents and entities. ║ ║ ║
║ ║ Actions are separated to ║ ║ ║
║ ║ use as a combined operations. ║ ║ ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 4 ║ Has pre-build entities like ║ Has pre-build entities like @sys.date, ║ Has pre-build entities ║
║ ║ temperature, number, URLs, ║ @sys.color, @sys.unit-currency… etc. ║ builtin.intent.alarm, ║
║ ║ emails, duration… etc. ║ ║ builtin.intent.calendar, ║
║ ║ ║ ║ builtin.intent.email… etc. ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 5 ║ Doesn’t have integration module ║ Has integration module to connect ║ Has integration to Microsoft ║
║ ║ to directly communicating with ║ directly to Facebook messenger and ║ Azure and other services, can be ║
║ ║ Facebook messenger or other ║ other messenger api’s. Has support for ║ deployable in any supported ║
║ ║ messenger APIs. but has web ║ deploying in to heroku server, enterprise ║ servers. ║
║ ║ service api to hook services. ║ paid environment. ║ ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 6 ║ Early in 2015, joined Facebook ║ Created by a team who built personal ║ LUIS was introduced together with ║
║ ║ and opened up the entire platform ║ assistant app for major mobile platforms ║ Microsoft Bot Framework and Skype ║
║ ║ to be free for both public and ║ with speech and text enabled conversations. ║ Developer Platform which can be ║
║ ║ private instances. ║ acquired by google (sept 2016). ║ used to create Skype Bots. ║
╠══════╬════════════════════════════════════╬═════════════════════════════════════════════╬════════════════════════════════════╣
║ 7 ║ Wit.ai API for developers of iOS, ║ Api.ai has SDKs for Android, iOS, ║ LUIS allow building applications ║
║ ║ Android, Node.js, Raspberry Pi, ║ the Apple Watch, Node.js, Cordova, ║ by using the LUIS web interface. ║
║ ║ Ruby, Python, C, Rust and ║ Unity, C#, Xamarin, Windows Phone, ║ No coding needed other than the ║
║ ║ Windows Phone. It even ║ Python and JavaScript. It also can be ║ ability to interpret and use the ║
║ ║ has a JavaScript plugin for ║ integrated with Amazon’s Echo and ║ returned JSON in application. ║
║ ║ front end developers. ║ Microsoft’s Cortana. ║ It is also possible to use the ║
║ ║ ║ ║ LUIS REST API for ║
║ ║ ║ ║ automation of applications. ║
╚══════╩════════════════════════════════════╩═════════════════════════════════════════════╩════════════════════════════════════╝
Update: API.AI is now Dialogflow. Learn more here.
Solution 2 - Artificial Intelligence
This blogpost has a really good analysis and comparison of Luis, Wit.ai, Api.ai, Amazon Alexa and IBM Watson services. It also has a nice background on why you would want to build a conversational bot in the first place and some of the challenges that come with that. It's written by the people behind YumiBot (a bot that gives you price quotes for app development).
The general gist is that Wit.ai and Luis are great choices if you're experimenting and just want to get something out for free. Api.ai has a great service and user experience but isn't free. Same with IBM Watson, the latter priced more for enterprise work. Alexa's API is great but only works with Alexa (but given that they have a huge userbase, isn't a bad deal).
Their advice is also to not rely too much on one provider:
> We would recommend you store all data needed for your model in a structured way in your own code repository. So later you can retrain the model from scratch, or even change the language understanding provider if needed. You just don’t want to be in a situation when a company shuts down their service and you are completely unprepared. Do you remember Parse?
I hope this helped a little! I think the best way to make a choice is to just give these services a try. Given that a lot of them are still heavy under development and adding features/changing pricing models, you should try coming at them with a specific use-case and see which one can get you to where you need the quickest.
Solution 3 - Artificial Intelligence
We have recently published an evaluation study of seven NLU API-enabled services: API.ai, Amazon Lex, Microsoft LUIS, IBM Watson Conversation, wit.ai, Recast.ai and Snips.ai.
A brief summary of our findings:
- IBM Watson intent detection is the best one, especially on smaller training datasets (although when trained on over 2000 samples the difference is indistinguishable).
- API.AI is free, the performance on big enough training set matches IBM Watson and Microsoft LUIS.
- Microsoft LUIS works significantly faster than others in our tests. wit.ai has somewhat worse performance and response time than the three above, but it’s free and it provides the best language coverage (some 50 languages).
- Amazon Lex has quite strict API limits (the training set size is limited to 200K symbols, which may be insufficient to reach a good intent detection quality for a multi-intent assistant; also it requires all training utterances to be labeled by entities, which complicated preparation of the dataset.
Solution 4 - Artificial Intelligence
One aspect of this question is how efficient are these tools at understanding natural language. In a recent benchmark we (Snips, a French AI company) just published, we have tested the built-in natural language engines of Alexa (Amazon), SiriKit (Apple), Luis (Microsoft), and API.ai (Google).
We tested their ability to understand natural queries like “Find me a salad bar I can go to for my lunch meeting”, “Order a cab for 6 people”, as well as 326 other queries.
The overall conclusion is that all solutions are imperfect.
More precisely, they all have similar levels of noise in their responses (between 60% and 90% precision), but there are significant differences in the breadth of language they can support. From this perspective, Luis performs the most poorly: on every use case we tested, it understood less than 14% of the queries. API.ai performs better, although not very reliably: it understands between 0 and 80% of the queries we tested, depending on the use cases. The highest levels of recall can be observed for Alexa (42% and 82% recall) and Siri (61% recall).
More details, and the raw data behind these results can be found in our blog post, Benchmarking Natural Language Understanding Systems
Solution 5 - Artificial Intelligence
i am going to answer the last part of your question around flexibility and being a developer, IMO it finally comes down to what you are looking for in these platforms.
If you are a developer using NodeJS or .Net, LUIS.ai has an extensive library and well defined code snippets and example to spin up a decent bot pretty quickly. The intents and entity recognition is a bit below par compared to google, but if you are Microsoft Shop, there are a lot of 1-click integrations to O365, Teams, Skype, cortana etc. The cons for LUIS.ai is their service seems very unstable, as of this writing their LUIS.ai website is not functioning rejecting connections and it has been more than a week, where cortana integration is not working. So the platform is still a work in progress.
Api.ai, from a pure NLU perspective is better than Luis.ai, the followup intents are very easy to setup, the speech priming is vastly superior to Luis.ai(even after speech priming). The cons i would say, it the connectabilty and also the API to build a bot are a bit more complicated than building an MSBot based chat bot.
Another platform open source platform that is gaining traction is RASA NLU. https://rasa.com/. Comparatively the entity recognition and the ranking is still a bit sketchy of large datasets, but its open sources and if you want to get your hands dirty, you can fork their github platform and improve it.
From a pure development perspective, its easier to fly up a chatbot in the MS platform(using luis.ai or qnamaker.ai ), but be prepared to have challenges as they work on stabilizing the platform.
-Kartik
Solution 6 - Artificial Intelligence
In my opinion Luis is more robust and can extract entities in different languages. I've tested in api.ai and dutch did not work for me. If you need english only then any one of them should be fine but if you need to support more languages then better test those languages as well before getting stuck with one service. Bing speech to text is ok but i think to get more robust solution you will need another microsoft service that cleans voice and noise.
Solution 7 - Artificial Intelligence
I was using DialogFlow but I switched to LUIS. Why? because when you call DetectIntent in DialogFlow you get a JSON with the selected intent and its confidence level but I need to get a list of intents with the confidence level of each one. The same happens with wit.ai and api.ai.
On the other hand, LUIS gives you a list of intents as a response. That way, I can apply further processing on my side.
This is an example from LUIS when you search for "book flight to Cairo" (part of the LUIS example):
{
"query": "Book me a flight to Cairo",
"topScoringIntent": {
"intent": "BookFlight",
"score": 0.9887482
},
"intents": [
{
"intent": "BookFlight",
"score": 0.9887482
},
{
"intent": "None",
"score": 0.04272597
},
{
"intent": "LocationFinder",
"score": 0.0125702191
},
{
"intent": "Reminder",
"score": 0.00375502417
},
{
"intent": "FoodOrder",
"score": 3.765154E-07
},
],
"entities": [
{
"entity": "cairo",
"type": "Location",
"startIndex": 20,
"endIndex": 24,
"score": 0.956781447
}
]
}
On the other hand, the UI to onfigure DialogFlow is much powerful than what you get with LUIS.