These are rush notes from Perflexity AI CEO Aravind Srinivas’s talk at the MIT IIA Conference (#MITforge2024).
He explains the Perplexity implementation, and many more interesting tidbits related to their early years and current plans.
How is Perplexity implemented?
- Perplexity.AI is not an LLM, and does not use its own LLM
- Instead, it is implemented as an LLM orchestrator
- LLMs are used as reasoning engines
- Perplexity harnesses LLM reasoning power using search indexes
- Basically, Perplexity implements AI search
- “It’s as if Chatgpt and Wikipedia had a child”
- Perplexity uses these other LLMs through their API
- Reality is that the user does not care which model you use behind the scenes
- The model is not the moat, but the orchestration is
- Perplexity makes sure answers are readable
- It presents answers differently depending on type of query
How does Perplexity orchestrate a query?
- One LLM model chunks the query, parallel LLM models run query chunks, one LLMmodel looks up/searches chunks, one LLM model aggregates the answer
Should companies operate on open source models?
Answer: Use the easiest possible LLMs, through APIs from LLM providers. Get the product out. Collect lots of data. Collect failures. Then, use open source. GPT is still expensive!
- Perplexity is B2C product now
- Lots people love Perplexity Pro.
- In general, there is skepticism from employers to let employees use Perplexity
- There are compliance requirements. These compliance requirements are higher for fine tuning, because they retain fine tuning data in their weights
- Databricks, Snowflake are well positioned for Data Warehouse for fine tuning
His time as a PhD student
- Did MIT accept Aravind Srinivas for PhD?
- Berkeley accepted him, MIT did not
What advice does Aravind Srinivas give to PhD students in AI?
- A PhD will prepare you for taking risks. You start by publishing 4-5 papers first year, taking ideas from advisor or other students. Then, you’ll want your own ideas
- His PhD started in RL. When GPT1 appeared, he talked to inventor, and got excited. But, at the time, GPT1 was laughed out.
- For him, 8 months followed without a paper. But, it was a rich time when he absorbed key things.
- PhD is not an even distribution,he says
Who did he pitch Perplexity to, initially?
- First pitch was to Elad Gil
- It was pitched as best way to disrupt Google – asking questions through a glass, with voice, using vision input. It was going to be a multimodal search.
- Vision was – imagine you can talk to any device. For example, that you go to the mall, and can ask questions of digital concierges around you.
- He hopes to still get to that vision
What is his opinion of GPU hardware?
- Nvidia hardware will be best for training
- For inference, answer is more subtle.
- You can pack more density per chip. Use more powerful chips, in the cloud – and fewer of them.
- At the edge, on devices, Google, and especially Apple have the best GPUs
- For inference, or at the edge – that’s where people can build new companies. For example: Grok, which does cloud inference. At the edge, you can compute with Nvidia.
What are Aravind Srinivas’s seed investments?
- His path was – Berkeley PhD, Open AI stint, then Perplexity. But he is also now seed investor in 2-3 companies, including Mistral.
- His philosophy is – if the product is good, invest. There is no time to research.
- For every startup, you can come out with reasons not to invest. Just go for it.
- For Mistral, and all of his investments – he knew some of the founders.There is lots of synergy between Perplexity and his investments.
Andrei Radulescu-Banu is Founder of Analytiq Hub. We develop data and AI workflows for healthcare, revenue cycle management, and robotics.