Open at Intel host Katherine Druckman chatted with Zilliz’s Tim Spann at last fall’s All Things Open conference about everything from his work with the Milvus project, to building AI applications the right way and what that might mean, all the way to spooky AI, and his project for capturing ghosts as unstructured data. Enjoy this transcript of their conversation.
“We tend to look at AI and think it’s trying to beat us. We look up and go “Oh, it’s horrible.” We have to be better teachers and mentors to our children, and to AI. Because in a sense, AI is our child. We have to treat it properly, then it’ll treat us properly.”
—Tim Spann, Principal Developer Advocate, Zilliz
Katherine Druckman: Hey, Tim. Thank you for joining me. We are here at All Things Open. Why don't you introduce yourself a little bit and tell us what you do.
Tim Spann: Sure. I'm Tim Spann. I'm principal developer advocate at Zilliz covering Milvus, the open source vector database. I'm based out of New York. We cover the whole east coast of the US.
The Importance of Community in AI
Katherine Druckman: I do want to mention, we have something in common. We both occasionally participate in the AI Alliance. How'd you get started with that?
Tim Spann: We joined because we're very community oriented. We run a community focused meetup where it's not about us. We don't put our name in it. It's anyone in AI, data, unstructured data.
Katherine Druckman: There's a lot of important work happening, and if we don't come together in these communities and have these important conversations, the work does not move forward in the way that benefits all of us. So, I wondered if you could tell us about the work that you do specifically with the Milvus project.
Tim Spann: Sure. I think it did bring up something important, because as opposed to other times in software, it is more important than ever that all different companies and communities, organizations, really come together over AI, because there's AI unregulated, and not regulated by us. We need the community from all different companies, research places, colleges working together on this, because if we just let some big companies drive it into their own place, we could be in trouble.
Katherine Druckman: You gave a presentation earlier. I'm actually giving a presentation tomorrow, and it's about what you just mentioned. And that it benefits us more if we work together than if we work in silos. We're repeating work, which is inefficient to begin with. And you cannot solve the toughest problems without all of the perspectives, or as many perspectives as you can get. Tell us a little bit about the talk you gave this morning.
Advanced RAG and Multimodal Models
Tim Spann: My talk was Advanced RAG. Last year, we had one of our speakers talk about the basics of RAG. And in that year, a lot has changed. It used to be that you're okay with just one simple AI model, one simple vector database to look stuff up, a little bit of text, some kind of glue code to put it together and you magically augmented your AI generation. But now, so many things have happened. And while we're talking about this, someone's coming out with a new style, someone is adding a new model…
Katherine Druckman: Right, a new product to new architecture.
Tim Spann: Yeah, it's incredible. The one that's coolest to me is, with these multimodal models, I can now do RAG with images and text together, and that's really cool. I put my cat in there and a description and I'm like, "Oh, now I get a super cat."
Katherine Druckman: Really quickly, I just wanted to make sure everyone listening knows RAG is retrieval augmented generation. We've talked about it on this podcast before, but just wanted to make sure. I was actually just at a really good workshop at Grace Hopper in Philadelphia all about multimodal RAG. It's just the options for building applications are getting more and more interesting, but also complex.
Tim Spann: That is a problem. Zilliz was at Grace Hopper remotely on the day before. We had a little hackathon going, which was awesome.
Katherine Druckman: It was big, and it was crowded, and it was a little overwhelming, but it was great. You had to have some really good walking shoes. Even more so than here. It’s very encouraging to see, especially that many women, I've never been in a space with that many women in technology in one place before ever.
Tim Spann: I think that's AI. AI has broken down some kind of barrier.
Katherine Druckman: AI development was definitely a hot topic, I will say that, for sure.
Tim Spann: I think we're in an interesting time again.
Katherine Druckman: Yeah. Isn't that a curse? May you live an interesting time?
Tim Spann: Yes. That is a Chinese curse. I lived through those times in the dot-com era. This feels different though.
Katherine Druckman: It feels like maybe when the internet was invented, or maybe the web. Or when web pages became something that everybody knew about and used. It feels like that.
Tim Spann: That same page too.
Katherine Druckman: AI's not new.
Tim Spann: No. But now it's useful for normal people.
Katherine Druckman: For the regular people, yeah.
Tim Spann: And for developers, and regular people. And with every fresh change, I find myself saying things like, "Oh, there's a new model," "Oh, there's a model," "Oh, there's a new free model," "Oh, it's doing this now, wow."
The Future of Agentic RAG
Katherine Druckman: What should app developers be the most excited about right now?
Tim Spann: I think the code assistants are helpful, but I think the new agentic RAG where you could use your coding skills to build out flows that make a real app. To be able to take information from different places with your own API's, your own vector database, multiple models, and code them together to do something. You can type in a prompt like, "I need to get to the Raleigh airport. I need to be there by this time. How should I do it?" The model is then able to look things up, hit your database, do an API call, look within your parameters to say, "Hey, don't spend more than this much money to figure this out. Come back to me when you get the top three things, and I could tell you if you like it, then you try again."
Being able to do that and have it act as an agent for me, and even far more complex things, I think that could be really cool for developers. Because they need people to code these together. As you said, things are more complex because there's more options. They say it's going to take away coding jobs. Yeah, maybe you're not going to write the 5,000 lines of input and check each value, you're going to be writing more interesting full apps that do things individually.
Katherine Druckman: You've explained it, but I wondered if you could just clarify a little bit what you mean by agentic RAG.
Tim Spann: With agentic RAG, we have the idea of agents. And the agents act independently, and they achieve something. It could be real simple: calling a function, or doing a lookup. And then you put them together and they just work for you. There's a lot more involved, but the simple idea of being able to have it do something for you outside of the bounds of just answering a question, or looking something up in a vector database, is pretty cool. Even better, you can have that scale out to as wide as you need to. I think right now though, I haven't found that one explosive agent library. I've seen a couple of new ones coming out the last couple of weeks, so I'm like, "I got to explore these." Which one's going to be the spring of agents?
Challenges and Excitement in AI Development
Katherine Druckman: There's always something coming out, right? And I hear you talk about, "I'm waiting for the next thing," "I'm waiting for the next thing." How do you keep up with all this stuff?
Tim Spann: I keep up with around 30 podcasts and 20 newsletters. I also put out my own newsletters. It’s a lot. I should write an agent to do it. Because right now I'm doing too much. I have a project and a demo to do, and I'll store that in my vector database for search and later can grab images and the text.
Katherine Druckman: Okay, so that's next week's project, right?
Tim Spann: How old is it now? It's like, "Oh, that's a month old. That can't be any good anymore." There's so much. While I was waiting for this to start, I saw three new models, and a new version of Llama 3.2. I wish someone would review them all as part of a show, like: "These are all the types of mac and cheese boxes. These are all the models that are out this week. Let's try them all."
Katherine Druckman: If that show does not exist, there's a show idea for somebody. I'm pretty sure it does though, but I don't know.
Tim Spann: Llama notebook and have it generate that podcast for me automatically.
Katherine Druckman: Have you ever actually listened to a podcast that was generated by AI?
Tim Spann: I tried one. I generated one myself, and it's very generic. It's interesting that it was not as horrible as I thought it would be.
Katherine Druckman: That's a good way of putting it. It's not as horrible. I actually heard someone suggest using it as an easy way to digest multiple news sources. It makes me a little sad, because as somebody who has made podcasts for several years, the human stuff is the fun stuff. It's the quirky stuff that is uniquely human. Can AI be quirky? Maybe, but not necessarily in the way that resonates.
Tim Spann: No. To me, it's not entertainment. It doesn't summarize it enough. I guess it'd be great if you could put an agent out that collects all the sources, then records a podcast, and then I can listen to it and go, "Okay, I could use that." Like everything else, we've got to figure out how to use these generated things to be tools for us. This is the new VI. Having an agent is now where VI was.
Instead of me going to VI to type something, okay, let me go launch an agent that's going to do 90% of my work, but I do that special human part. I get it all together and make sure it's right.
Katherine Druckman: I had that conversation just a little while ago actually, about when we held up a mirror to ourselves. That's a lot of what all this generative AI is, right? It's trained on stuff we put out into the world. And it's not necessarily the best. It may be a broken or dirty mirror that we're holding up to ourselves. But it is still a mirror in a way.
Tim Spann: Well, maybe this is a reflection to say we have to do better. We tend to look at AI and think it’s trying to beat us. We look up and go “Oh, it’s horrible.” We have to be better teachers and mentors to our children, and to AI. Because in a sense, AI is our child. We have to treat it properly, then it’ll treat us properly.
Building AI the Right Way
Katherine Druckman: I think the original premise here is: build things the right way. What does that mean to you? A lot of people are in a position where they're building a lot of generative AI apps because somebody's told them they need to. "We need to add some AI to this," that's a common thread. So what does that mean? What does building it the right way mean to you?
Tim Spann: I think there's a couple things. First is really looking for a community to collaborate with. AI inherently has to be under human control, so you have to work with other people, work with open communities, open source; things like meetups, talk with other people, so you get an idea of, how can we do this that it's good and get input from a lot of different people.
And then use any tool you can find available. There's a couple of open source guardrails. There's making sure you check out the models before you use them. HuggingFace is a good place for that. Use as many open models that have good things in them. I've added extra layers of models that do things like filter out hate speech. Doing that can slow you down, but it’s better in the long run. Use something like a vector database. So, if I'm going to have something that's building an app, I do it with information I know and trust as much as possible.
Katherine Druckman: So, what would you like people to know? I mean, your day job is working with the Milvus community, correct? What is it that you really want people to know about the project?
Tim Spann: It’s one of the oldest vector databases coming out of research, and we’ve learned things the hard way in open source. We started off as a simple project designed to run on a couple of nodes, because who thought it would ever get this big? And then quickly, we find out, "It's going to get bigger." So we made it cloud native, separated it out to be very distributed in a fully distributed architecture, and then worked with the communities around the world, so they had all the features that people wanted. And people wanted flexibility because models keep changing. So support any kind of model, support different type of vector search implementations, and have that all in a flexible manner where you can run it in a notebook, run it in Docker, run it in Kubernetes, run it in a SAS, wherever you need to run it at any scale, up to billions of vectors, or for people starting out in your notebook with 10 documents. And make it as easy as you can to get started, but have the flexibility to put in a different type of model, put in a hybrid search, do advanced RAG techniques, and do multimodal. And whatever comes next, we work with the AI Alliance, we're part of the Linux Foundation.
Fun with AI: Capturing Ghosts
Katherine Druckman: You're part of OPEA, by the way.
Tim Spann: We're part of everything. We love everybody. We love Intel and all the other makers of stuff.
Katherine Druckman: Well, I think that kind of vendor-neutral community environment is really important, especially when you're dealing with emerging technology and pioneering technology. It's so important.
Tim Spann: Yeah. Because change is going to be the only constant for a while. So we don't try not to have anything that can't be swapped out. By the way: recently, I've been using Milvus to capture ghosts for Halloween.
Katherine Druckman: Okay, tell me about that.
Tim Spann: So, I figured out that ghosts are unstructured data.
Katherine Druckman: That makes sense.
Tim Spann: Because they don't have a structure, but they have attributes. And so I've got images of them, I've got attributes, I've got magnetic readings, I've got radiation readings, light readings, heat readings, I got a thermal camera. Actually, convenient to you, I'm using my OpenVINO camera inside the trap. It captures as much of that in all the different dimensions. We store that in Milvus as a multimodal model, and now I can search it. So I have a catalog of all these ghosts that I have captured. So when a new one comes along, I set up an app so people can report them. So you go, "Hey, is this a ghost?" And I could match it with a similarity search with text or the image, or both, and tell you, "No, that's probably not a ghost." Or, "Hey, that's AI," or, "Hey, that's Scooby-Doo." Whatever it is. Or, "Hey, you put a sheet on top of your cat. I'm going to save that anyway, but that's not a ghost."
Katherine Druckman: That's awesome.
Tim Spann: So I'm working on it. So, if people want to do research around Unstructured storage of ghosts, reach out to me.
Katherine Druckman: Spooky AI.
Tim Spann: Spooky AI. We have it.
Katherine Druckman: Awesome. Well, on that note, I really appreciate it. Thank you so much for stopping by and sitting down with me. And have a good rest of your show.
Tim Spann: Thank you. You too.
Katherine Druckman: You've been listening to Open at Intel. Be sure to check out more about Intel’s work in the open source community at Open.Intel, on X, or on LinkedIn. We hope you join us again next time to geek out about open source.
About the Guest
Tim Spann, Principal Developer Advocate, Zilliz
Tim Spann is a principal developer advocate for Zilliz and Milvus. He works with Apache NiFi, Apache Kafka, Apache Pulsar, Apache Flink, Flink SQL, Apache Pinot, Trino, Apache Iceberg, DeltaLake, Apache Spark, Big Data, IoT, Cloud, AI/DL, machine learning, and deep learning. Tim has over 10 years of experience with the IoT, big data, distributed computing, messaging, streaming technologies, and Java programming. Previously, he was a principal developer advocate at Cloudera, developer advocate at StreamNative, principal dataflow field engineer at Cloudera, a senior solutions engineer at Hortonworks, a senior solutions architect at AirisData, a senior field engineer at Pivotal, and a team leader at HPE. He blogs for DZone, where he is the Big Data Zone leader, and runs a popular meetup in Princeton & NYC on big data, cloud, IoT, deep learning, streaming, NiFi, the blockchain, and Spark. Tim is a frequent speaker at conferences such as ApacheCon, DeveloperWeek, Pulsar Summit, and many more. He holds a BS and MS in computer science.
About the Host
Katherine Druckman, Open Source Security Evangelist, Intel
Katherine Druckman, an Intel open source security evangelist, hosts the podcasts Open at Intel, Reality 2.0, and FLOSS Weekly. A security and privacy advocate, software engineer, and former digital director of Linux Journal, she's a long-time champion of open source and open standards. She is a software engineer and content creator with over a decade of experience in engineering, content strategy, product management, user experience, and technology evangelism. Find her on LinkedIn.