DSPy: The End of Prompt Engineering - Kevin Madura, AlixPartners

Thanks everybody for joining. I'm here to talk to you today about DSPy. Feel free to jump in with questions or anything throughout the talk. I don't plan on spending the full hour and a half or so. I know it's the last session of the day, so keep it casual. Feel free to jump in. I'll start with a little bit of background. Don't want to go through too many slides. I'm technically a consultant, so I have to do some slides, but we will dive into the code for the latter half. And there's a GitHub repo that you can download to follow along and play around with it on your own.

How many people here have heard of DSPy? Almost everyone. That's awesome. How many people have actually used it kind of day-to-day in production or anything like that? Three. Okay, good. So hopefully we can convert some more of you today. So high level DSPy, this is straight from the website. It's a declarative framework for how you can build modular software. And most important for someone like myself, I'm not necessarily an engineer that is writing code all day, every day. As I mentioned before, I'm more of a technical consultant. So, I run across a variety of different problems. Could be an investigation for a law firm. It could be helping a company understand how to improve their processes, how to deploy AI internally. Maybe we need to look through 10,000 contracts to identify a particular clause or paragraph. And so DSPy has been a really nice way for me personally and my team to iterate really quickly on building these applications. Most importantly building programs. It's not iterating with prompts and tweaking things back and forth. It is building a proper Python program and DSPy is a really good way for you to do that.

So I mentioned before there's a repo online if you want to download it now and kind of just get everything set up. I'll put this on the screen later on. But if you want to go here, just download some of the code. It's been put together over the past couple days. So, it's not going to be perfect production level code. It's much more of utilities and little things here and there to just come and kind of demonstrate the usefulness, demonstrate the point of what we're talking about today and we'll walk through all of these different use cases. So sentiment classifier going through a PDF some multimodal work a very very simple web research agent detecting boundaries of a PDF document you'll see how to summarize basically arbitrary length text and then go into an optimizer with Jeepo but before we do that just again kind of level set the biggest thing for me personally DSPy is a really nice way to decompose your logic into a program that treats LLMs as a first class citizen.

So at the end of the day, you're fundamentally just calling a function that under the hood just happens to be an LLM and DSPy gives you a really nice intuitive easy way to do that with some guarantees about the input and output types. So of course there are structured outputs, of course there are other ways to do this, Pyantic and others. But DSPy has a set of primitives that when you put it all together allows you to build a cohesive modular piece of software that you then happen to be able to optimize. We'll get into that in a minute.

So, just a few reasons of why I'm such an advocate. It sit at it sits at this really nice level of abstraction. So, it's I would say it doesn't get in your way as much as a Langchain. And that's not a knock-on Langchain. It's just a different kind of paradigm in the way that DSPy is structured. Allows you to focus on things that actually matter. So you're not writing choices zero messages content. You're not you're not doing string parser. You're not doing a bunch of stuff under the hood. You're just declaring your intent of how you want the program to operate, what you want your inputs and outputs to be. Because of this, it allows you to create computer programs. As I mentioned before, not just tweaking strings and sending them back and forth. You are building a program first. It just happens to also use LLMs.

And really the the most kind of important part of this is that and Omar the KB the founder of this or the the original developer of it had this really good podcast with A16Z. I think it came out just like two or three days ago. But it he put it a really nice way. He said it's a it's built with a systems mindset and it's really about how you're encoding or expressing your intent of what you want to do most importantly in a way that's transferable. So the the design of your system, I would imagine, or your program isn't going to move necessarily as quickly as maybe the model capabilities are under the hood. when we see new releases almost every single day, different capabilities, better models and so DSPy allows you to structure it in a way that retains the control flow retains the intent of your system, your program while allowing you to bounce from model to model to the extent that you want to or need to.

Convenience comes for free. There's no parsing, JSON, things like that. It again, it sits at a nice level of abstraction where you can still understand what's going on under the hood. If you want to, you can go in and tweak things, but it allows you to to kind of focus on just what you want to do while retaining the level of precision that you that I think most of us would like to have in and kind of building your programs. As mentioned, it's robust to kind of model and paradigm shifts. So, you can again keep the logic of your program. But it but keep that those LLMs infused in basically in line.

Now that being said, you know, there are absolutely other great libraries out there. Pedantic AI, Langchain, there's many many others that allow you to do similar things. Agno is another one. This is just one perspective and it may not be perfect for your use case. For me, it took me a little bit to kind of gro how DSPy works and you'll see why that is in a minute. So, I would just recommend kind of have an have an open mind, play with it. Run the code, tweak the code, do whatever you need to do. And just see how it might work, might work for you. And really, this talk is more about ways that I found it useful. It's not a dissertation on the ins and outs of every nook and cranny of DSPy. It's more of, you know, I've run into these problems myself now. I naturally run to DSPy to solve them. And this is kind of why. And the hope is that you can extrapolate some of this to your own use cases.

So we we'll go through everything fairly quickly here, but the core concepts of DSPy really comes down to arguably five or these six that you see on the screen here. So we'll go into each of these in more detail, but high level signatures specify what you want the L what basically what you want your function call to do. This is when you specify your inputs, your outputs. Inputs and outputs can both be typed. And you defer the rest of the basically the how the implementation of it to the LLM. And we'll see how we how that all kind of comes together in a minute. Modules themselves are ways to logically structure your program. They're based off of signatures. So, a module can have one or more signatures embedded within it in addition to additional logic. and it's based off of pietorrch and some of the in terms of like the methodology for how it's structured and you'll you'll see how that comes to be in a minute.

Tools we're all familiar with tools MCP and others and really tools fundamentally as DSPy looks at them are just Python functions. So it's just a way for you to very easily expose Python functions to the LLM within the DSP kind of ecosystem if you will. Adapters live in between your signature and the LLM call itself. I mean, as we all know, prompts are ultimately just strings of text that are sent to the LLM. Signatures are a way for you to express your intent at a at a higher level. And so, adapters are the things that sit in between those two. So, it's how you translate your inputs and outputs into a format basically explodes out from your initial signature into a format that is ultimately the prompt that is sent to the LLM. And so, you know, there's some debate or some research on if certain models perform better with XML as an example or BAML or JSON or others. And so adapters give you a nice easy abstraction to to basically mix and match those at at will as you want.

Optimizers are the most interesting and for whatever reason the most controversial part of DSPy. That's kind of the first thing that people think of or at least when they hear of DSPy they think optimizers. We'll see a quote in a minute. It's not optimizers first. It is just a nice added benefit and a nice capability that DSPy offers in addition to the ability to structure your program with the signatures and modules and everything else. Metrics are used in tandem with optimizers that that basically defines how you measure success in your in your DSPy program. So the optimizers use the metrics to determine if it's finding the right path if you will.

So signature as I mentioned before it's how you express your intent your declarative intent can be super simple strings and this is the weirdest part for me initially but is one of the most powerful parts of it now or it can be more complicated class-based classbased objects if you've used pyantic it that's basically what what it runs on under the hood. So this is an example of one of the class-based signatures. Again, it it's basically just a pyantic object. What's super interesting about this is that the the names of the fields themselves act almost as like mini prompts. It's part of the prompt itself. And you'll see how this comes to life in a minute. But what's ultimately passed to the model from something like this is it will say okay your inputs are going to be a parameter called text and it's based off of the name of the that particular parameter in this class. And so these things are actually passed through. And so it's it's very important to be able to name your parameters in a way that is intuitive for the model to be able to pick it up. You can add some additional context or what have you in the description field here.

So most of this, if not all of this, yes, it is proper, you know, typed Python code, but it's also it also serves almost as a prompt ultimately that feeds into the model. And that's basically translated through the use of adapters. And so just to highlight here like these, it's the ones that are a little bit darker and bold, you know, those are the things that are effectively part of the prompt that's been sent in and you'll see kind of how DSPy works with all this and formats it in a way that again allow you to just worry about what you want. Worry about constructing your signature instead of figuring out how best to word something in the prompt.

So the question for folks online is what if I already have a great prompt? I've done all this work. I'm a I'm a amazing prompt engineer. I don't want my job to go away or whatever. Yes. So, you can absolutely start with a custom prompt or something that you have demonstrated works really well. And you're exactly right that's that can be done in the doc string itself. There's there's some other methods in order for you to inject basically system instructions or add additional things at certain parts of the ultimate prompt and or of course you can just inject it in the in the final string anyway. I mean it's just you know a string that is constructed by VSPI. So absolutely this doesn't necessarily prevent you it does does not prevent you from adding in some super prompt that you already have. Absolutely. And to your point it is it can serve as a nice starting point from which to build the rest of the system.

Here's a shorthand version of the same exact thing which to me the first time I saw this so this was like baffling to me. But it it that's exactly how it works is that you're basically again kind of deferring the implementation or the logic or what have you to DSPy and the model to basically figure out what you want to do. So in this case, if I want a super super simple text sentiment classifier, this is basically all you need. You're just saying, okay, I'm going to give you text as an input. I want the sentiment as an integer as the output. Now you probably want to specify some additional instructions to say okay your sentiment you know a lower number means negative you know a higher number is more positive sentiment etc. But it just gives you a nice kind of easy way to to kind of scaffold these things out in a way that you don't have to worry about like you know creating this whole prompt from hand. It's like okay I just want to see how this works and then if it works then I can add the additional instructions then I can create a module out of it or you know whatever it might be. It's these shorthand or it is this shorthand that makes experimentation and iteration incredibly quick.

So modules it's that base abstraction layer for DSPy programs. There are a bunch of modules that are built in and these are a collection of kind of prompting techniques if you will and you can always create your own module. So to the question before, if you have something that you know works really well, sure yeah, put it in the module. That's now the kind of the base assumption, the base module that others can build off of. And all of DSPy is meant to be composable, optimizable, and when you deconstruct your business logic or whatever you're trying to achieve by using these different primitives, it all it's intended to kind of fit together and flow together. And we'll get to optimizers in a minute, but at least for me and my team's experience, just being able to logically separate the different components of a program, but basically inlining LLM calls has been incredibly powerful for us. And it's just an added benefit that at the end of the day, because we're just kind of in the DSPy paradigm, we happen to also be able to optimize it at the end of the day.

It comes with a bunch of standard ones built in. I don't use some of these bottom ones as much, although it's they're super interesting. The base one at the top there is just DSpi.predict. That's literally just, you know, an LM call. That's just a vanilla call. chain of thought probably isn't isn't as relevant anymore these days because models have kind of ironed those out but it is a good example of the types of kind of prompting techniques that can be built into some of these modules and basically all this does is add some some of the strings from literature to say okay let's think step by step or whatever that might be same thing for react and codeact react is basically the way that you expose the tools to the model. So, it's wrapping and doing some things under the hood with basically taking your signatures and it's injecting the Python functions that you've given it as tools and basically React is how you do tool calling in DSP.

Program with thought is pretty cool. It kind of forces the model to think in code and then we'll return the result. And you can give it a, it comes with a Python interpreter built in, but you can give it some custom one, some type of custom harness if you wanted to. I haven't played with that one too too much, but it is super interesting. If you have like a highly technical problem or workflow or something like that where you want the model to inject reasoning in code at certain parts of your pipeline, that's that's kind of an really easy way to do it. And then some of these other ones are basically just different methodologies for comparing outputs or running things in parallel.

Here's what one looks like. Again, it's it's fairly simple. It's, you know, it is a Python class at the end of the day. And so you do some initial initialization up top. In this case, you're seeing the the shorthand signature up there. So, I'm this module just to give you some context is an excerpt from one of the the Python files that's in the repo is basically taking in a bunch of time entries and making sure that they adhere to certain standards, making sure that things are capitalized properly or that there are periods at the end of the sentences or whatever it might be. that's from a real client use case where they had hundreds of thousands of time entries and they needed to make sure that they all adhere to the same format. This was one way to to kind of do that very elegantly, at least in my opinion, was taking up top you can define the the signature. It's adding the some additional instructions that were defined elsewhere and then saying for this module the the change tense call is going to be just a vanilla predict call. And then when you actually call the module, you enter into the forward function which you can inter basically intersperse the LLM call which would be the first one and then do some kind of hard-coded business logic beneath it.

Tools as I mentioned before these are just vanilla kind of Python functions. It's the DSP's tool interface. So under the hood, DSPy uses light LLM. And so there needs to be some kind of coupling between the two, but fundamentally any type of tool that would that you would use elsewhere, you can also use in in DSPy. And this is probably obvious to most of you, but here's just an example. You have two functions, get weather, search web. You include that with a signature. So in this case, I'm saying the signature is I'm going to give you a question. please give me an answer. I'm not even specifying the types. It's just going to infer what that means. I'm giving it the get weather and the search web tools and I'm saying, okay, do your thing, but only go five rounds just so it doesn't spin off and do something crazy. And then a call here is literally just calling the React agent that I created above with the question, what's the weather like in Tokyo? We'll see an example of this in the code session, but basically what this would do is give the model the prompt, the tools, and let it do its thing.

So adapters, before I cover this a little bit, they're basically prompt formatterers, if you will. So the description from the docs probably says it best. It's you know it takes your signature the inputs other attributes and it converts them into some type of message format that you have specified or that the adapter has specified and so as an example the JSON adapter taking say a pyantic object that we defined before this is the actual prompt that's sent into the LLM and so you can see the input fields so this would have been defined as okay clinical note type string patient info as a patient details object object which which would have been defined elsewhere and then this is the definition of the patient info. It's basically a JSON dump of that pantic object.

So this idea there's like a base adapter default that's good for most cases and this is if you want to tweak that to do something more specific. That's right. Yeah. The question was if if there's a base adapter and would this be an example of where you want to do something specific? Answer is yes. So it's a guy pashant who is I have his Twitter at the end of this presentation but he's been great. He did some testing comparing the JSON adapter with the BAML adapter. And you can see just intuitively even even for us humans the way that this is formatted is a little bit more intuitive. It's probably more token efficient too just considering like if you look at the messy JSON that's here versus the I guess slightly better formatted BAML that's here. can actually improve performance by you know five to 10 percent depending on your use case. So it's a good example of how you can format things differently. The the rest of the program wouldn't have changed at all. You just specify the BAML adapter and it totally changes how the information is presented under the hood to the LLM.

Multimodality. I mean this obviously is more at the model level but DSPy supports multiple modalities by default. So images, audio, some others. And the same type of thing, you kind of just feed it in as part of your signature and then you can get some very nice clean output. This allows you to work with them very, very, very easily, very quickly. And for those eagle-eyed participants, you can see the first lineup there is attachments. It's probably a lesserk known library. Another guy on Twitter is awesome. Maxim, I think it is. he created this library that just is basically a catch-all for working with different types of files and converting them into a format that's super easy to use with LLMs. He's a big DSPy fan as well. So he made basically an adapter that's specific to this. But that's all it takes to pull in images, PDFs, whatever it might be. You'll see some examples of that and it just makes at least has made my life super super easy.

Here's another example of the same sort of thing. So this is a PDF of a form 4 form. So, you know, public SEC form from Nvidia. Up top I'm just giving it the link. I'm saying, okay, attachments, do your thing. Pull it down, create images, whatever you're going to do. I don't need to worry about it. I don't care about it. This is super simple rag, but basically, okay, I want to do rag over this document. I'm going to give you a question. I'm going to give you the document and I want the answer. And you can see how simple that is. Literally just feeding in the document. How many shares were sold? Interestingly here, I'm not sure if it's super easy to see, but you actually have two transactions here. So, it's going to have to do some math likely under the hood. And you can see here the thinking and the the ultimate answer.

Is it on the rag step? Is it creating a vector store of some kind or creating embeddings and searching over those? Is there a bunch going on in the background there or what? This is poor man's rack. I should have clarified. This is this is literally just pulling in the document images and I think attachments will do some basic OCR under the hood. But it doesn't do anything other than that. That's it. All we're feeding in here, the the actual document object that's being fed in, yeah, is literally just the text that's been OCRD. the images, the model does the rest.

All right, so optimizers let's see how we're doing. Okay. Optimizers are super powerful, super interesting concept. It's been some research that argues I think that it's just as performant if not in cert in certain situations more performant than fine-tuning would be for certain models for certain situations. there's all this research about in context learning and such. And so whether you want to go fine-tune and do all of that, nothing stops you. But I would recommend at least trying this first to see how far you can get without having to set up a bunch of infrastructure and, you know, go through all of that. See how the optimizers work. But fundamentally what it allows you to do is DSPy gives you the primitives that you need and the organization you need to be able to measure and then quantitatively improve that performance.

And I mentioned transferability before. This the transferability is enabled arguably through the use of optimizers because if you can get okay I want to I have the classification task works really well with 41 but maybe it's a little bit costly because I have to run it a million times a day. Can I try it with 41 nano? Okay, maybe it's at 70% whatever it might be. But I run the optimizer on 41 nano and I can get the performance back up to maybe 87%. maybe that's okay for my use case, but I've now just dropped my cross my cost profile by multiple orders of magnitude. And it's the optimizer that allows you to do that type of model and kind of use case transferability, if you will. But really all it does at at the end of the day under the hood is iteratively prompt iteratively optimize or tweak that prompt, that string under the hood. And because you've constructed your program using the different modules, DSPy kind of handles all of that for you under the hood. So if you compose a program with multiple modules and you're optimizing against all that, it it by itself DSPy will optimize the various components in order to improve the input and output performance.

And we'll we'll take it from the man himself, Omar. You know, ESPI is not an optimizer. I've said this multiple times. it's it's just a set of programming abstractions or a way to program. You just happen to be able to optimize it. So again, the value that I've gotten and my team has gotten is mostly because of the programming abstractions. It's just this incredible added benefit that you are also able to to should you choose to to optimize it afterwards. And I was listening to this to Dwaresh and and Carpathy the other day and this kind of I was like prepping for this talk and this like hit home perfectly. I was thinking about the optimizers and someone smarter than me can can ple you know please correct me but I think this makes sense because he he was basically talking about using LLM as a judge can be a bad thing because the model being judged can find adversarial examples and degrade the performance or basically create a situation where the judge is not not scoring something properly. because he's saying that the model will find these little cracks. It'll find these little spirious things in the nooks and crannies of the giant model and find a way to cheat it. Basically saying that LM as a judge can only go so far until the other model finds those adversarial examples.

If you kind of invert that and flip that on its head, it's this property that the optimizers for DSpir are taking advantage of to optimize to find the nooks and crannies in the model, whether it's a bigger model model or smaller model to improve the performance against your data set. So that's what the optimizer is doing is finding finding these nooks and crannies in the model to optimize and improve that performance.

So a typical flow, I'm not going to spend too much time on this, but fairly logical constructor program which is decomposing your logic into the modules. You use your metrics to define basically the contours of how the program works and you optimize all that through to to get your your your final result.

So, another talk that this guy Chris Pototts just had maybe two days ago, where he made the point, this is what I was mentioning before, where Jeepa, which is, uh, you probably saw some of the the talks the other day, where the optimizers are on par or exceed the performance of something like GRPO, another kind of fine-tuning method. So, pretty impressive. I think it's an active area of research. people a lot smarter than me like Omar and Chris and others are are leading the way on this. But point being I think prompt op prompt optimization is a pretty exciting place to be and if nothing else is worth exploring.

Then finally metrics again these are kind of the building blocks that allow you to define what success looks like for the optimizer. So this is what it's using and you can have many of these and we'll see examples of this where again at a high level your program works on inputs it works on outputs the optimizer is going to use the metrics to understand okay my last tweak in the prompt did it improve performance it did it degrade performance and the way you define your metrics provides that direct feedback for the optimizers to work on. So here's another example, a super simple one from that time entry example I mentioned before. So they can be the metrics can either be like fairly rigorous in terms of like does this equal one or or you know some type of equality check or a little bit more subjective where using LLM as a judge to say whatever was this generated string does it adhere to these you know various criteria whatever it might be but that itself can be a metric and so all of this is to say it's a very long-winded way of saying in my opinion this is probably most if not all of what you need to construct arbitrarily complex workflows, data processing pipelines, business logic, whatever that might be. Different ways to work with LLMs. If nothing else, DSPy gives you the primitives that you need in order to build these modular composable systems.

So, if you're interested in some people online, there's many many more. There's a Discord community as well. But usually these people are are on top of the latest and greatest and so would recommend giving them a follow. You don't need to follow me. I don't really do much, but the others on there are are really pretty good.

Okay, so the fun part, we'll actually get into some to some code. So, if you haven't had a chance, now's your last chance to get the repo.

Okay. So, I'll set up Phoenix, which is from Arise, which is basically an obser an observability platform. I just did this today, so I don't know if it's going to work or not, but we'll we'll see. We'll give it a shot. But basically what this allows you to do is have a bunch of observability and tracing for all the calls that are happening under the hood. We'll see if this works. We'll give it like another 5 seconds. But it should, I think, automatically do all this stuff for me.

All right. So, something's up. Okay, cool. So, I'll just I'm just going to run through the notebook, which is a collection of different use cases, basically putting into practice a lot of what we just saw. Feel free to jump in any questions, anything like that. We'll start with this notebook. There's a couple of other more proper Python programs that we'll walk through afterwards. But really the intent is a rapidfire review of different ways that DSPy has been useful to me and others. So load in the end file. Usually I'll have some type of config object like this where I can very easily use these later on. So if I'm like call like model mixing. So if I have like a super hairy problem or like some workload I know will need the power of a reasoning model like GPD5 or something else like that, I'll define multiple LM. So like one will be 41, one will be five, maybe I'll do a 41 nano you know Gemini 2.5 flash, stuff like that. And then I can kind of intermingle or intersperse them depending on what I think or what I'm reasonably sure the workload will be. and you'll see how that comes into play in terms of classification and others. I'll pull in a few others here. I'm I'm using open router for this. So, if you have an open router API key, would recommend plug plugging that in. So, now I have three different LLMs I can work with. I have Claude, I have Gemini, I have 41 mini. And then I'll ask basically for each of them who's best between Google Anthropic OpenAI. All of them are hedging a little bit. They say subjective, subjective, undefined. All right, great. It's not very helpful. But because DSPy works on Pyantic, I can define the answer as a literal. So I'm basically forcing it to only give me those three options and then I can go through each of those. And you can see each of them, of course, chooses their own organization. The reason that those came back so fast is that DSP has caching automated under the hood. So as long as nothing has changed in terms of your your signature definitions or basically if nothing has changed this is super useful for testing it will just load it from the cache. So I ran this before that's why those came back so quickly. But that's another kind of super useful piece here.

Make sure we're up and running. So, if I change this to hello with a space, you can see we're making a live call. Okay, great. We're still up. So, super simple class sentiment classifier. Obviously, this can be built into something arbitrarily complex. Make this a little bit bigger. But I'm basically I'm giving it the text, the sentiment that you saw before, and I'm adding that additional specification to say, okay, lower is more negative, higher is more positive. I'm going to define that as my signature. I'm going to pass this into just a super simple predict object. And then I'm going to say, okay, well, this hotel stinks. Okay, it's probably pretty negative. Now, if I flip that to I'm feeling pretty happy. Good thing I'm not in a hotel right now. you can see I'm feeling pretty happy. Comes down to eight. And this might not seem that impressive and you know it's it's not really but the the the important part here is that it just demonstrates the use of the shorthand signature. So I have I have the string, I have the integer, I pass in the custom instructions which would be in the dock string if I use the class B classbased method.

The other interesting part or or useful part about DSPy comes with a bunch of usage information built in. So because it's cached, it's going to be an empty object. But when I change it, you can see that I'm using Azure right now, but for each call, you get this nice breakdown. and I think it's from late LLM, but allows you to very easily track your usage, token usage, etc. for observability and optimization and everything like that. Just nice little tidbits that are part of it here and there. Make this smaller. We saw the example before in the slides, but I'm going to pull in that form 4 off of online. I'm going to create this doc objects using attachments. You can see some of the stuff it did under the hood. So, it pulled out PDF plumber. It created markdown from it. Pulled out the images, etc. Again, I don't have to worry about all that. Attachments make that super easy. I'm going to show you what we're working with here. This case, we have the form four. And then I'm going to do that poor man's rag that I mentioned before. Okay, great. How many shares were were sold in total? It's going to go through that whole chain of thought and bring back the response.

That's all well and good, but the power in my mind of DSPy is that you can have these arbitrarily complex data structures. That's fairly obvious because it uses paidantic and everything else, but you can get a little creative with it. So in this case, I'm going to say, okay, a different type of document analyzer signature. I'm just going to give it the document and then I'm just going to defer to the model on defining the structure of what it thinks is most important from the document. So in this case, I'm defining a dictionary object and so it will hopefully return to me a series of key value pairs that describe important information in the document in a structured way. And so you can see here again this is probably cached but I passed in I did it all in one line in this case but I'm saying I want to do chain of thought using the document analyzer signature and I'm going to pass in the input field which is just the document here. I'm going to pass in the document that I got before. And you can see here it pulled out bunch of great information in the super structured way. And I didn't have to really think about it. I just kind of deferred all this to the model to DSPy for how to do this.

Now, of course, you can do the inverse in saying, okay, I have a very