“By combining algorithms, data, knowledge and human intelligence, we believe we can give machines an unprecedented power to help society solve its grand challenges.”
Tell us about your journey. How did you get the idea behind SparkBeyond?
Since I was a child, I dreamed of building a machine that could think, create and produce new ideas – all on its own. Fast forward a few decades, and here I am at SparkBeyond, doing exactly that.
SparkBeyond’s story began in 2013, where a moonshot experiment led to our quest to develop a machine that could come up with novel and creative ideas. This machine can be curious, ask questions, and test hypotheses and theories on data. It can then prove and refute those same hypotheses, or connect the dots across rows of complex data, and then articulate its ideas and insights in plain English. This is the premise of our first core product, SparkBeyond Discovery.
In building a platform that generates millions of novel theories and hypotheses, we first had to understand the very components, or building blocks, of an idea. For a computer, those building blocks take the form of code, data and knowledge. So, just like search engines crawl the web for text, our machine started indexing all the code, data and knowledge on the web. If our machine explores billions of data elements and algorithmic combinations in order to express novel ideas, we were hopeful that some of those ideas would lead to the next breakthrough.
The majority of investments in AI today are based on replicating the human mind’s ability to perceive reality, whether it’s vision, speech, language understanding, robotics. SparkBeyond is focused on the complete opposite end of the spectrum, which is giving machines the capacity to ask questions, to come up with ideas, to create – the creative part of AI.
We view SparkBeyond Discovery as a platform to make a planetary-scale impact on the world. When we launched six years ago, we asked ourselves, “How can we apply AI in a very different way than how it is applied today? Can it invent? Can it ask questions? Can it create?” Those were the questions we wanted to answer.
Read More: ITechnology Interview with Umesh Sachdev, Co-founder and CEO at Uniphore
What challenges do data scientists and researchers come across that SparkBeyond is helping overcome?
Human ideation is inherently limited by cognitive bottlenecks and biases, which restrict us in generating and testing ideas at scale and high throughput. We’re also limited by the speed at which we can communicate. We don’t have the capacity to read and comprehend the thousands of scientific articles and patents published every day.
What’s more, the questions we ask are biased by our experience and knowledge, or even our mood.
In data science and research workflows, there are key bottlenecks that limit what a person or team can accomplish while working on a problem within a finite amount of time. For example, when exploring for useful patterns in data, a data scientist only has time to conceive, engineer, and evaluate a limited number of distinct hypotheses, leaving many areas unexplored.
Another challenge is an organization’s own data. This internal data may only reveal part of the story, whereas augmented external data sources can provide valuable contextual information. Without it, hypotheses based only on internal data don’t take into account the influence of external factors, such as weather and local events, or macro-economic factors and market conditions.
Instead, by mapping out the entire spectrum of dynamics that happen on earth, SparkBeyond’s products connect the dots between every data set that exists and offers a comprehensive viewpoint.
SparkBeyond offers more than one product. Can you break down the different technologies?
Yes.
SparkBeyond Discovery is an automated analytics platform for citizen data scientists and analysts. Its AI technology allows analysts to rapidly and exhaustively discover useful patterns, predictive features, and novel insights within and across all different types of datasets – including an organization’s internal first-party data, as well as a wide range of contextual, external datasets. The platform is built atop a type of ‘Hypothesis Engine’, which integrates the world’s largest collection of algorithms and bypasses human cognitive bias to produce millions of hypotheses and questions in minutes. These hypotheses ensure that any meaningful signals in the data are surfaced. Then, these signals are often immediately actionable, and can be used as predictive features in machine learning models.
The second product is our Research Studio platform, which, unlike SparkBeyond Discovery, bridges the gap in data-scarce environments by infusing web-scale knowledge and other unstructured data. It leverages AI to intelligently mine the vast ocean of human knowledge found in public domains, spanning the web, research papers, patents, news, clinical trials etc. — allowing a researcher to direct complex queries and find multi-hop relationships using a number of under-the-hood techniques such as NLP. What’s powerful about this Research Studio is that it is an advanced question-and-answering machine – when presenting a query, or a series of queries, it actually gives you answers, unlike other engines that would present a list of sources from which the researcher needs to extract the insights themselves.
AI and automation seem to be powering everything around us these days. Can you tell us about that side of SparkBeyond?
From the very beginning, we set out to create a platform that could harness humanity’s collective intelligence and generate its own hypotheses using AI. By combining algorithms, data, knowledge and human intelligence, we believe we can give machines an unprecedented power to help society solve its grand challenges.
Just like search engines crawl the web for text, our machine started indexing the code, data and knowledge on the web, and amassed one of the world’s largest libraries of open-source code functions. Using both automation and AI, our platform employs these functions to generate four million hypotheses per minute—a capacity that allows the technology to work through hundreds of good and bad ideas every second. We believe that we simply can’t afford to wait for the next eureka moment, and challenges like climate change, sustainability, cancer and other persistent problems can be solved with technology that accelerates breakthroughs.
AI is often used to replicate the cognitive functions of humans, not to come up with creative ideas. And that’s what we’re using it for. We start with a problem, look at its root causes, explore existing solutions and deficiencies within those solutions. Working through that process allows you to identify opportunities for innovations. In other words, you’re presented with what the world says about your problem and what the data says. The overlap is extremely interesting.
Our platforms are multilevel and show you the distribution of sources it pulls information from—a sort of visual representation of its thought process. The results are ranked according to the number of sources that agree and there is a credibility score for each source that is also taken into account. You can blacklist or exclude certain sources if you like and overlay different data streams to find new correlations.
Read More: Mini, Micro and Nano Apps: What 2022 Looks like for IT DevOps Industry
SparkBeyond specifically did some work around COVID-19. Can you tell our audience about that?
When the pandemic hit, we were presented with an opportunity to leverage our technology to benefit the greater good. Our hypothesis engine was able to take reams of data gathered by governments and prove and disprove thousands of hypotheses, providing real solutions to lower infection rates. In Italy, for example, our analysis of the data showed that there were higher rates of infections in certain areas, including drinking fountains in parks, gas stations, railway stations and tourist hotspots. One of the major insights was that the A1 highway, which runs from Milan to Naples through the middle of the country, was a particularly focused area for infections, just before the lockdown was implemented.
Argentina was another country that reached out to us for Covid-19 intelligence. Our platform was put to use in this case in helping decide how citizens can return to work and minimize the devastating effect lockdowns have on the economy. And these were not the only countries we were able to help. We also provided solutions to Singapore and Israel.
Ultimately, we received calls from governments around the world saying, “Can you help us?” That was obviously gratifying and galvanizing for the team, but these things are only as good as your impact. Our platform is unique because it’s designed around massive challenges like overcoming the pandemic, where people are highly driven to make a difference.
What is your vision for the space 5-10-20 years from now?
Our vision includes the possibility of AI-driven evidence-based policy making, as it could lead to the next generation of democracy. It could lead to an interesting alternative to our current system, which decades ago we simply did not have the means to do.
We also like to think about a time in the not too distant future when our platform could take the last 100 years of inventions and extract universal principles that could be used to solve modern issues. Imagine a tool that could draw insights from a hypothesis engine and attempt to create new inventions. Sure, many of the ideas generated might be half-baked and nonsense, but they would still provide useful starting points for innovators to be inspired by.
Look at the United Nations’ Sustainable Development Goals as just one example. Could we use the invention machine to come up with millions of ideas or solutions that could be sorted and improved with the help of experts by mining the knowledge of human intelligence? At SparkBeyond, we’re harnessing mankind’s collective intelligence to solve the world’s current challenges using different sources of intelligence and we see the vast potential to take on even bigger challenges in the future.
Read More: ITechnology Interview with Kim Perell, CEO at 100.co
Thank you, Sagie! That was fun and we hope to see you back on itechnologyseries.com soon.
[To participate in our interview series, please write to us at sghosh@martechseries.com]
Sagie is an entrepreneur, technological visionary and machine intelligence enthusiast. Striving to bridge the gap between human & machine reasoning and interaction.Sagie devotes his spare time to advising exciting start-up companies and investors.
For the past 17 years, busy solving exciting challenges, while creating new ones in the spare time.
Passionate about computational knowledge representation, acquisition, storage, reasoning and processing.
Sagie has been lucky to serve in executive technological positions in disruptive start up companies.
SparkBeyond has built an automated research engine, capable of asking questions and discovering complex patterns in data. The technology is designed to leverage masses of data that exist on the web, intelligently augment your data and discover complex patterns. These patterns then become powerful tools for explaining, predicting and optimizing outcomes.
