Draft:Robust Intelligence
Submission declined on 1 October 2024 by Jamiebuba (talk).
Where to get help
How to improve a draft
You can also browse Wikipedia:Featured articles and Wikipedia:Good articles to find examples of Wikipedia's best writing on topics similar to your proposed article. Improving your odds of a speedy review To improve your odds of a faster review, tag your draft with relevant WikiProject tags using the button below. This will let reviewers know a new draft has been submitted in their area of interest. For instance, if you wrote about a female astronomer, you would want to add the Biography, Astronomy, and Women scientists tags. Editor resources
|  |
.png){kind=logo} | |
| Founded | 2019 |
|---|---|
| Headquarters | San Francisco, California, U.S. |
Key people |
|
Number of employees | 51-100 |
| Website | robustintelligence |
Robust Intelligence is an artificial intelligence (AI) security company headquartered in San Francisco, California.[1] The company’s platform protects organizations from the security and safety risks of artificial intelligence models and applications.
Robust Intelligence was founded in 2019 by Dr. Yaron Singer, a Gordon McKay Professor of Computer Science and Applied Mathematics at Harvard University, and Kojin Oshiba, a machine learning researcher and Harvard University alumnus.[2][3] In September 2024, Robust Intelligence was acquired by Cisco.[4]
History
[edit]Robust Intelligence was co-founded by Yaron Singer, a tenured professor of Computer Science and Applied Mathematics at Harvard, and Kojin Oshiba in 2019 after nearly a decade of combined robust machine learning research at the university and Google Research. Recognizing the state of artificial intelligence adoption and the chronic challenges of AI risk in industry, the pair developed the industry’s first AI firewall.
Prior to founding Robust Intelligence and his ten-year tenure at Harvard, Yaron worked as a Postdoctoral Research Scientist at Google on the Algorithms and Optimization team. This role followed receiving his PhD in computer science from University of California, Berkeley in 2011.[5]
Co-founder Kojin Oshiba graduated with a Bachelor’s Degree in Computer Science and Statistics from Harvard University in 2019. During this time, he spent a year as a machine learning engineer at QuantCo and helped co-found the company’s Japan branch.[6]
Robust Intelligence emerged from stealth mode in 2020 with the announcement of its $14 million fundraising round led by Sequoia Capital.[3] The company raised a $30 million Series B fundraising round in 2021 led by Tiger Global, with participation from Sequoia, Harpoon Venture Capital, Engineering Capital, and In-Q-Tel.[7]
Dr. Hyrum Anderson, Robust Intelligence’s Chief Technology Officer, joined the company in 2022 from Microsoft where he co-organized the AI Red Team and served as the chair of its governing board. An accomplished machine learning and cybersecurity expert, Anderson co-founded the Conference on Applied Machine Learning in Information Security (CAMLIS) and co-authored the book Not With a Bug, But With a Sticker: Attacks on Machine Learning Systems and What To Do About Them.[8]
Several notable figures and technologies in the field of artificial intelligence have emerged from research and development that began at Robust Intelligence. Most prominent are LangChain, an open source framework designed to simplify the creation of applications using LLMs, developed by former Robust Intelligence machine learning engineering leader Harrison Chase; and LlamaIndex, a data framework for connecting custom data sources to LLMs developed by Jerry Liu.[9][10]
In September 2024, Cisco acquired Robust Intelligence.[4]
Research
[edit]Robust Intelligence researchers have identified and responsibly disclosed several adversarial machine learning techniques and AI security vulnerabilities, both while working at the company and in academia. Recent findings include:
- Tree of Attacks with Pruning: A new, efficient jailbreak method that works across foundation models that was co-developed with researchers from Yale University.[11]
- Llama Prompt-Guard-86M Exploit: A simple method for bypassing Meta’s detection model using only spaces between characters.[12]
- OpenAI Structured Output Exploit: An exploit which bypasses the safety mechanisms, including refusal capabilities, of OpenAI’s Structured Mechanisms functionality.
- OpenAI Indirect Prompt Injection: A susceptibility in LLMs to malicious instructions concealed within external sources, such as video transcripts or a provided document.[13]
- Google Gemini Jailbreak: With algorithmic prompt improvement, researchers could bypass safety guardrails in the Google Gemini model.[14]
- Real Attackers Don’t Compute Gradients: An exploration of the gap between adversarial machine learning research and real-world attackers.[15]
- NVIDIA NeMo Guardrails Exploits: A jailbreak which overcomes Nvidia’s NeMo guardrails to elicit sensitive data leakage and other harmful outcomes.[16]
- Poisoning Web-Scale Training Datasets is Practical: Research into data poisoning for models trained on large and indiscriminate datasets from the open internet.[17]
References
[edit]- ^ "Tiger Global leads $30M round for AI reliability startup Robust Intelligence". SiliconANGLE. 2021-12-09. Retrieved 2024-09-30.
- ^ "Robust Intelligence to expand Israel activities after raising $30m". Globes. 2021-12-12. Retrieved 2024-09-30.
- ^ a b Cai, Kenrick. "This Harvard Professor And His Students Have Raised $14 Million To Make AI Too Smart To Be Fooled By Hackers". Forbes. Retrieved 2024-09-30.
- ^ a b Kovacs, Eduard (August 27, 2024). "Cisco to Acquire AI Security Firm Robust Intelligence". SecurityWeek.
{{cite news}}: CS1 maint: url-status (link) - ^ Tardif, Antoine (2022-03-09). "Yaron Singer, CEO at Robust Intelligence & Professor of Computer Science at Harvard University – Interview Series". Unite.AI. Retrieved 2024-09-30.
- ^ "Kojin Oshiba". Forbes. Retrieved 2024-09-30.
- ^ Lardinois, Frederic (2021-12-09). "Robust Intelligence raises $30M Series B to stress test AI models". TechCrunch. Retrieved 2024-09-30.
- ^ Kumar, Ram Shankar Siva; Anderson, Hyrum (2023-03-31). Not with a Bug, But with a Sticker: Attacks on Machine Learning Systems and What To Do About Them. John Wiley & Sons. ISBN 978-1-119-88399-9.
- ^ Palazzolo, Stephanie. "Exclusive: AI startup LangChain taps Sequoia to lead funding round at a valuation of at least $200 million". Business Insider. Retrieved 2024-09-30.
- ^ "LlamaIndex". Forbes. Retrieved 2024-09-30.
- ^ "Researchers Use AI to Jailbreak ChatGPT, Other LLMs". www.darkreading.com. Retrieved 2024-09-30.
- ^ Claburn, Thomas (July 29, 2024). "Meta's AI safety system defeated by the space bar". The Register.
{{cite news}}: CS1 maint: url-status (link) - ^ Burgess, Matt. "The Security Hole at the Heart of ChatGPT and Bing". Wired. ISSN 1059-1028. Retrieved 2024-09-30.
- ^ Wiggers, Kyle (2023-12-07). "Early impressions of Google's Gemini aren't great". TechCrunch. Retrieved 2024-09-30.
- ^ ""Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice". Informatics@KCL. 2023-01-18. Retrieved 2024-09-30.
- ^ "Subscribe to read". www.ft.com. Retrieved 2024-09-30.
{{cite web}}: Cite uses generic title (help) - ^ "It doesn't take much to make machine-learning algorithms go awry". The Economist. ISSN 0013-0613. Retrieved 2024-09-30.
- in-depth (not just brief mentions about the subject or routine announcements)
- reliable
- secondary
- strictly independent of the subject
Make sure you add references that meet all four of these criteria before resubmitting. Learn about mistakes to avoid when addressing this issue. If no additional references exist, the subject is not suitable for Wikipedia.