In the summer of 2014, fresh out of high school, I started an internship at a startup tech company called Synapsify.
I worked on something called the qtree>, but in order for you to understand its importance, I need to explain what my company does. Or if you want, you can just go to the qtree> page.
What is Synapsify?
“Synapsify is an award-winning technology company that builds applications that semantically reads and learns from written content similar to humans, for accelerated discovery, insight and recommendations. Synapsify CORE, its voice of customer analytical tool, helps marketers, researchers and insight professionals get to the heart of their customer’s stories faster and more accurately, saving them time, money and resources. ”
A more informal description for those of you unfamiliar with trends in technology:
Synapsify is a text analytics company using ground breaking methods to interpret valuable meaning from human generated text.
I don't think I'm doing the company justice with my description here, but I loved working there and I think their work with text analytics is amazing.
The Story
Lawrence Au, my mentor and CTO, explained to me the power of phonemics in my first interview.
And I can't tell the story as well as he can but I'll do my best....
It started with a failed attempt at A.I. (Artificial Intelligence).
And like most programmers, Lawrence and his team were doing well until they hit a wall:
They built an A.I with the disposition of an autistic child.
Stumped, he paid a visit to a friend of his who was a psychiatrist.
First Insight
"How do you cure children with autism?" - Lawrence
"Well you can't cure them. But you can at least get them to function better by teaching them how to understand human emotion." - Psychiatrist
Second Insight
One day, Lawrence took a cab in a foreign country. The cab driver spoke Arabic. The funny part is that these cab drivers often make deals with street vendors. They sometimes advertise souvenirs to the tourists they drive around.
The cabbie would speak to Lawrence in guttural English, "You like vases?"
"No."
Cabbie goes back to speaking in arabic on the phone.
Lawrence could pretty much guess what he was saying purely on his tone, "Dang, he doesn't want a vase, you got anything else???"
The Point?
Although Lawrence had no idea what the cabbie was saying, he could gain an insight into the flow of the conversation based purely on phonemics.
And thus, Lawrence decided that he would help his autistic, child-like A.I understand human emotion through phonemics alone.
Which is ground breaking, and just SO COOL!
Some cool stuff his program has achieved:
- It took the reading portion of the SAT and scored a 530. With no vocabulary understanding.
- In a book contest with over 1,000 submissions, his program was able to select the top 15 based purely on analyzing emotion and tension from the text. Some of the top 15 later became published.
- Search and filter the most meaningful reviews among millions of Walmart product responses.
- Rate books and help novice writers improve their craft in minutes.
It's pretty flexible!
I don't think I did his achievements any justice, and I'm sure there are tons of amazing things I didn't get to describe. Most of the complexities of Synapsify went over my head at the time- I was a high schooler whose only education in programming was AP Compsci in sophmore year.
Now onto Why we need qtree>....(What i did, yay!)
Synapsify is a text-analytics company. One of its main jobs is to process insanely large amounts of text.
Thus, one of its main objectives is to prioritize speed, especially in parsing text.
For example, one of our valuable key points is the fact that machines can read and process faster than humans. Ingram, a large publishing company, is one of our biggest customers. Ingram currently pays people to read books, summarize them, and rate them. In contrast, Synapsify's services would be invaluable if all that could be done within minutes.
See? Time is of the essence.
However, Common Lisp, the language we use, is not an easy language to optimize.
Lisp is awesome. Lisp is great for a.i
Buuuttt it's not the best in terms of speed.
However it's flexible, it's got MACROS, and it's unbelievably concise. It's a beautiful language, and I'm proud to consider it as my first language in programming.
Good ol' C....
And Lisp...should be respected. It should not be underestimated.
Genius is crazy, no?
qtree>:
- A preprocessor that tokenizes and efficiently stores input text.
- qtree> implements a sophisticated radix tree algorithm customized for natural language parsing.
- qtree> uses ground-breaking asymmetric storage/retrieval design to simplify parsing of dictionary entries from natural language texts.
- qtree> can efficiently stores text and map relationships between meaning and words.
Head onto the qtree> page for more!