Taylor It was this research study that Crimson turned me onto a while ago where particularly men always need like a thing, Like for you and me to hang out, theoretically, you and I need an activity, like something to do together.
Sean Mm, yeah.
Taylor Like we're gonna watch the game together, we're gonna drink beer together, we're gonna go fishing together, we're gonna do a podcast together. Rarely do males of our demographic just sit down and shoot the breeze. Like, how are you? What's going on in life? What are you doing? that's normally how I conduct most of my management conversations anyway, tell me more about I'm more interested about you and what's going on in your life than I am about the work, frankly, which is
Sean Mm-hmm.
Taylor weird. we probably wouldn't be having this conversation if it weren't for the fucking podcast. Like, you haven't called me in years just to shoot the breeze, Sean. You know?
Sean No, no, no.
Taylor We need to get back on the phone like we're fifteen year old, you know, nineteen ninety five curly cord
Sean It's
Taylor stretched in from the kitchen. You're like hiding out in the back door. Anyway.
Sean it's so true. Even even say being at the pool this summer, in theory the pool is the third thing, but then I'm just sitting there watching kids with other adults and I'm like, what am I what am I doing? I'm so bored.
Taylor You're soaking up life. No, it's so good. It's so good for you to just sit there and not the best part about the pool is it's so frickin' hot and sunny. It's not like you could be on a screen. I mean, I have taken my laptop to the pool sometimes to work in the late
Sean Yeah, true.
Taylor afternoon when it's like a you know, not terribly hot afternoon and the kids need to get out of the house and I need to get some work finished, you know, at the end of the day. But it's not like the greatest experience. You can barely see the damn screen. The smudges
Sean Yeah, it is true.
Taylor show up. my god, the smudges on my screen. It's
Sean The smudges.
Taylor the worst. Anyway. Well, we should probably get started here. What you think?
Sean Yeah, let's do it.
Taylor Okay. well, what is this? Episode twelve? Welcome to episode twelve
Sean I think that's right.
Taylor of Blood, Sweat and Tokens, the podcast where me and Sean here are exploring, you know. Product tooling, product development, agentic coding, workflows, things like that in the age of natural language processing. Sean, how you doing this week?
Sean I'm doing all right. in between travels at the moment, so my my brain is struggling a little bit. But I yeah, yeah, that's that's me. We're mid summer. It's gonna close up really fast here. What you just did some traveling too, yeah.
Taylor I did. We just got back from a a lake trip to Michigan. we're kind of in the middle of nowhere. It was sorta weird, but it was also great. I I always enjoy those trips because there's not a whole lot to do, you know. So you're just sort of
Sean Mm-hmm.
Taylor forced to hang out with each other. which my kids I'm not sure they love it, but me and my wife enjoy
Sean Yeah.
Taylor that. We kind of keep captive and and trap you know, they can't escape when when you're out in the middle of nowhere. Sounds terrible. Anyway.
Sean It's true. So it was a an inland lake?
Taylor Yeah, it was have you ever you know Michigan. You've got don't you you guys vacation in Michigan. Michigan
Sean Yeah, we're up there all the time.
Taylor is like the land of t ten bajillion lakes. I mean, it's freaking crazy. Every time you turn around or every other block, there's like another s substantial body of water with beautiful real estate flanking all corners. And some of are kind of hard to get into if you don't own the property. So you know, we like to go fishing, we like to swim, we like to get a pontoon boat. This time we we got a a really fancy pontoon boat that that could pull the kids in an inner tube for a day. It was obscenely expensive, but it was so fun. and I and I saw the coolest pontoon boat ever. You know, pontoon boats are always you know, they're awesome when you're on them, but they're like kind of clunky and weird. I never understand how they cost so much money. But we saw one on the lake up there. It was a sea do. You know Sea Doo? They the jet skis?
Sean Mm-hmm.
Taylor Okay, so it was like Sea Dew brand pontoon boat. It was called the C-Doo Switch. Man, it looked so cool. Like, okay, so first of all, it uses the same water propulsion system as the traditional. Yeah,
Sean That's what I was gonna ask. Yeah.
Taylor so it doesn't have like an outboard motor hanging off the back, which I'm sure probably makes it less, you know, it's like the Mac to the PC of pontoon. Right. Like it's less you can't swap out your own outboard or like do individual components. I'm assuming this is like the John Deere of pontoons, right? Like your right to replace and your right to fix is probably significantly hindered by the beautiful, fully self-contained design of this jet propulsion system on the C-Doo switch. But anyway, now I'm like, dude, I want one of these. I it's the most
Sean Yeah.
Taylor impractical thing in the world. They start at like 30 grand, which is not cheap, but as far as like the ease of use. Now I've always in the outboard, always I always worry about it, right? Cause you gotta trim it up and down so you don't like crank it on across something on the bottom of the lake. you know, I'm worried about my my son. I don't know, a few years ago we we were on a lake trip and he jumps off the boat to like something fell out into the water and he like freaked out and was like, I gotta grab it. And he jumped out in front of the boat as the boat was moving forward. And of course we just all immediately like Freaked out. And I mean it was fine. It turned out okay, but it was also kind of a boner move. And anyway, the thought of having Yeah,
Sean You're like, yeah. science is gonna have
Taylor I don't know. Anyway, sorry, I'm off on the tangent as per usual. What are we talking about today? So I've got a couple of really interesting things that I want to talk about in the AI space. gave you a little teaser of that earlier this week or last week, I think. So we'll get to that shortly, but in the meantime. How's the product going? What do you got anything from a demo or have you made any progress? How much how much how much stuff has gotten accomplished in the last week?
Sean I've made good progress. And so I was I was thinking earlier today about what to show because what we what we to recap, what we showed on the last episode was kind of the end-to-end flow of someone saying, I want this content to change, and what that looks like within this application. And what we're trying to do is then add in in the same seamless process, the ability to adjust code as well. And so that's what I've been working on. But rather than showing half a demo, I wanted to I wanted to throw the the hurdle I'm currently running into at you and see what your take is and how you would approach the resolution to it. Because I have an I have a picture in my mind, but I'll just I'll I'll I'll paint the the scenario. So Now, okay, user user goes in and asks for a change. And if it's a text change, we're like, okay, cool. And again, to recap, what the system is doing is it is creating a duplicate record and putting it in this staging environment. And we just have one staging/slash preview environment today. We'll eventually have multiples. And so when the preview loads up, the it is looking for all of the content that applies to the current page. And then if there's any content that is in preview mode, that's going to override what's on the current page and it will show that when you're in preview mode. And so that's how the content piece works. And it can work seamlessly because it's all server side generated. But with the code, what we're going to do is actually change the code. And so the the ag there's one agent that is sort of kind of the orchestrator task manager. And so that person is going to or that person, that agent is going to get that request in and then say, this is not a this is not a text change, this is code change. And if you're like, okay, cool, it's gonna let's do it, you know, in s instead of hey, change this. headline you say make this form center aligned. It's like actually I need to add an attribute to this component. So then it kicks off the process it probab it's it's essentially just a c a claude code request to go adjust that component open a pull request which is going to kick off a deploy preview right and then all of that Is there sitting in the in the pull request? And that's where I am today. And so I got this, you get this flow working, right? And then it's like, okay, cool. Here's your here's your preview. And I've actually got it rendering in the screen. And it's wrong. And it's wrong every time. And the reason it's wrong is not because the agent didn't do the work, because the agent did the work. So I think f contact form has no alignment attribute. And this request come in comes in and you're like, I need to I want to center everything in the contact form. And so then the agent does that work and it adds the attribute. But It's just an ability, like it's an attribute for that component. There needs to be something else that tells the page that tells
Taylor Yeah, you need to update the component to take advantage of the prop that got passed into.
Sean Right, and that part is content, right? So, the value, yes. Mm-hmm.
Taylor Sure. sorry, the value of the prop is content, but the code change under is what is inside of the actual component itself.
Sean Yes. So that's kind of an interesting conundrum, right? Because you've got this pull request on a branch with a deploy preview, and you need a specific set of content that should come from the database to actually be able to preview what shows up on screen, but we don't want the content In theory, you don't want that content to be publishable outside of the context of the pull request, right? Because if it's it in in theory, that could cause some other repercussions if you're changing names of attributes or s you know, if there is a value and I you know, it's gonna mess something up for one reason or another in production. So how would you couple those things together?
Taylor Well, the last time The last time we talked, you I asked you the question of like how do you let's back up. In a previous conversation, you showed me the page and then you made adjustments to elements through your voice language CMS integration. Or like update. I think it was like the founders link in the nav from this to that or whatever. And that led to Yeah, that'd be helpful.
Sean Yeah, and I can pull this up so we've got something to look at too. But
Taylor That that led to a conversation about like how are you managing The state across multiple changes within a singular thing. And then we looked at a Kanban board that you built that showed every individual change. I think that's right. And then you showed me the database and you explained that there was block instances. And what you were doing was you were duplicating a block instance, and a page could be made up of an amalgamation of different instances of blocks that are on that page.
Sean Mm-hmm.
Taylor And the instance is kind of a pointer. to where that lives in the hierarchy of the changes or like the different iterations of that block as you have made adjustments to them through this interface. Is there not a world where you could effectively just extend that model and give the block instance some sort of awareness about which pull request, branch, deployment context that it lives within? And it should be deployed within. I this is very similar to the way Netlify's infrastructure works, right? Like you have
Sean Mm-hmm.
Taylor deploy contexts and you can have values for something, in this case, an environment variable using the Netlify analogy. You have environment variables that differ across different contexts. One of those contexts is your production context, one of those contexts is your development context, or sorry, your your staging site, right?
Sean Mm-hmm.
Taylor one of those contexts could be the deploy preview environment. One of those contexts is your local environment. you know, if you implement I said staging environment, but like if you were to implement like a branch deployments, I think is what you guys call them, where you have a separate build that's tied to each individual branch, you can have contexts that are unique to each of those branches. Maybe there's something in that pattern that's worth focusing on. You know, because you've already got the infrastructure to iterate on individual instances of the content objects themselves. And it sounds to me like your problem is there is a content change that is also coupled to this pull request. And if you kind of considered that just the context is this pull request, the minute that gets merged, you presumably whatever if you made like a schema adjustment. Like this is part of the reason I I think this is getting complicated because. You've introduced a new prop to a component that requires now a new content value. Right?
Sean Yes.
Taylor Okay, so if you're using a structural, like a SQL style bat, like a relational database, usually you would have individual fields that represent each one of those content values.
Sean Mm-hmm.
Taylor By adding a new content value on the fly through the direction you've given to the CMS. You're in some ways requiring yourself to either add a new column to that table or use some sort of doc store or like JSON blob, which I think you told me last time you're using a JSON blob to manage a lot of this on the block instance itself.
Sean Yeah, so it's it's theoretically you sh you wouldn't need a migration. Yeah.
Taylor Then I think the real question would be like spend some time on that, you know, that spend some time on the method that would determine here are the blocks that belong in this context. Like what defines a context? You know, is there a user interface to manage this, or is it smart enough to just manage it on its own? The minute that you create a pull request. You know, does that constitute the context or is the context like live across different environments? Like you have a you have a context of what you're working on, you're working locally. Is that a global, globally accessible context? Like you can also the same exact context lives in deploy preview? Or, you
Sean Mm-hmm.
Taylor know, one of the problems I've had pardon me with Netlify and the way that you guys do deployment context is that there's no easy way to like promote From one environment to the other, at least I've not found that. Like, for example, if I create a pull request and I add a new environment variable to support a new API key that I need for the code that's being introduced in that pull request. when I merge that, it would be amazing if I could just click a button and have that environment variable go along with the pull request. Like internal to Netlify, if you could just take the encrypted value for that API key out of the
Sean Mm.
Taylor pull requests context and move that in to the production level context, you've saved me a step. Like honestly,
Sean Yep, yep.
Taylor I find the whole management of environment variables to be extremely brittle. Sorry, I that's way off topic from what we're talking about here. We'll have to table that for another conversation. That said, I don't know, what do you think about that? Like considering those block instances as grouped logically within contextual relevancy.
Sean Yes. Yeah. So what I've been talking about as environment is essentially it's just a a specific type of context. I think I I agree. I think there's some some question in terms of what what is the anchor of that context. You know, it's so I I now have the concept over here of chat threads. So we can save some history and we can open a thread. So is the thread the context? Is the pull request the context? I'm not sure. Might just take some playing around with it to see what works well.
Taylor Mm-hmm. All right. well, that's an interesting challenge. Don't envy you having to solve that problem 'cause that sounds
Sean Ha ha.
Taylor that sounds complicated.
Sean yeah, so I'll I'll tinker with that and I'll plan on bringing that back and we'll it'll it'll all be solved and perfect next time.
Taylor I'm sure. I'm sure it will. All right. well, hey, listen, I wanna I had some questions that I wanted to ask you about like recent
Sean Yeah.
Taylor you know, current events in the AI landscape. And one of them I mentioned to you it well, I'm gonna deviate just a bit to begin with, but have you heard of Chat GPT or sorry, GPT Live? Are you familiar with that term? Does that come across your radar yet? Okay, so
Sean I don't think so. I don't think so.
Taylor GPT Live was announced by OpenAI a couple of weeks ago. And it is basically the first AI suite of AI models that introduce what's called full duplex architecture. And the reason, and I guarantee you might have noticed this, right? So we've talked obviously the whole intent of, or at least one of the kind of hypothetical or like or or the foundation of this podcast has a lot to do with natural language, right?
Sean Mm.
Taylor The ability to talk to computers and have a computers talk back to you. So the GBT live models are effectively a next evolutionary step in that regard. Because what it does is it allows like win it allows for this concept of full duplex architecture where it can basically think and act concurrently, similar to the way the human brain works, right? So like right now, in this moment, you're sitting here talking to me, you're probably processing all sorts of stuff. all around,
Sean Mm-hmm.
Taylor you know, like I can hear the air conditioner, my feet are cold, you know, my elbow is itchy because I get a bug bite. We're having this conversation. I'm focused on you. You have a big mustache and a bald head and I don't, I have a lot of hair. You know, these are just like random thoughts that are like floating through my through floating
Sean Yep. Yep.
Taylor through my brain as we talk. GBD Live is designed to basically facilitate a richer voice interaction because it can process multiple things concurrently. And if you think about it in the way that teams work or agent swarms, which we talked about a little bit a while ago, you described in a prior episode, you described your workflow as effectively having a delegation or an orchestration agent that then delegates tasks down to other sub-agents that can then work kind of independently and report back up. That is exactly how GPT Live works. So when you're talking to it, it can actually it has like an orchestrator layer that can talk and interact with you. But can delegate tasks in that voice conversation to other subagents. Things like in-depth reasoning or research, web searches, stuff like that can all happen concurrent to the conversation that you're having. and it's most obviously evidenced by if you open the voice assistant in Chat GPT today and you start talking to it, it's gonna be like, hmm, okay, you know, let me let me look that up. Like it is d it is morphed into this weird. two-way feedback loop where you know prior it was very transactional. You know, you'd open voices such and you'd be like, hey, where's the closest Walgreens? You know, and it would be like, well, let me think about that. And you'd hear go boop, boop, boop, you know, and it would go off. It was like the
Sean Mm, mhm.
Taylor audible version of a loading indicator. It would go off and do some research, come back with an answer. Well, it doesn't do that anymore. Now it'll be like, hmm, all right. And it's it it represents a very fluid very human-like communication medium. Interesting stuff. Have you played with this? Have you noticed that behavior on voice assistant? I don't know what you're probably exclusively clawed at this point, but have you played around at all with the voice assistant on ChatGPT recently?
Sean So I've I've spent more I've spent more time with Claude. I've gone back and forth a little bit recently, but it's interesting you're saying this and I'm I'm kind of I'm doing the full duplex thing while we're talking about it. I'm I'm reading about it and I d I just looked up whether Claude follows that same model because I had a question that I'm surprised it hasn't come up for us previously because we've talked about voice mode and I've been wondering w Because okay, let me back up for just a second. One of one of those tools that I'm building to help me with my everyday tasks, one of the agents that I'm trying to build out is a research agent, and I want to have an interactive talk mode with it. And as I've been researching more about how to build that, I've also been talking with Claude and ChatGPT. And one of those things I noticed was like, it is. It is really fast. I'm talking for like five minutes and then there is almost no latency and everything is already processed. So it's somehow
Taylor Uh-huh.
Sean it's not just speech to text process return to me. Something is happening. What's going on?
Taylor No. It's delegation. It's delegating. It's doing exactly what your workflow has kind of, you know, orchestrated the the sort of delegation you know, through things like, you know, sub agents and stuff like that. One cool the marketing campaign for this is awesome. It just came out. It's so funny. it's these old women. man,
Sean I see this video on the yeah.
Taylor it is hysterical. It is so well done. I'm I don't know. I've been really impressed with a lot of the marketing stuff on this. I mean, they have got some serious dollars and as a kind of you know, we live in the digital marketing space at Ample and it's I give anything to play with some of these campaigns and and learn a little bit more about both the products but also, you know, come up with some of those creative outcomes. I I thought this yeah, the the old lady video is hysterical.
Sean Alright, that's what I'm I'm watching after the episode. But are so wait the background or the sub agents or whatever, background tasks, are you saying it's still using speech to text technology? It's just delegating it? Or is it s something entirely different?
Taylor I I mean, I don't know, right? Because I'm not an engineer at OpenAI, but yeah, that's my assumption is that it's not even like when you look at this, yes, the full duplex thing is net new functionality, like being able to process in real time. Like one of the examples it gave is live translation, which is really cool. And they in that marketing video, they do an example of this. But the and and I always think about this, like how do translators at the UN, like how do they do that? thing where they process
Sean Yeah, yeah.
Taylor the input and somehow capture that in short-term memory while they're translating the output and they're always like you know five seconds behind the speaker and somehow they can keep up. That is fascinating to me. I will tell
Sean Mm. Yep.
Taylor you one thing about this GPT Live. I found it off putting that it interrupts me. I don't know if this is because I've got a teenager and an eight year old in house, but my kids interrupt like we have this major issue in our relationship. Me and my wife we'll try to have a conversation about anything can be anything. What we're gonna eat for dinner, how we're gonna pay the yard guy, you know, like whether we should buy that, you know Chest freezer, you name it. Like there's a million different conversations that happen. And every time we get in these conversations, some child walks into the room and just blurts out whatever's top of mind. You know? And it's always really sweet and endearing, but it's also just like, are you serious? Like we're having this conversation. So we're 13 years into this experience of having little humans that like to show up and interrupt. And I think it's driving us both completely insane. Here's a funny anecdote. I don't know if I've told you this before. Atticus, my son, will call me on the phone. And he, it's the weirdest thing. You know how like you make a phone call, you've been calling people your whole life. You pick up the phone, you dial the number, you put it to your ear. The other person picks up the receiver and greets you. They say, hello. Salutations. Like, welcome to my end of the phone call, right? On your end, you say, Hey, Sean. What's going on, buddy? And that's how conversations work. My 13-year-old,
Sean Typically, yep.
Taylor I don't understand it, but he has developed this thing where the amount of time it takes for me to pick up the phone and put it to my ear is n it is too lengthy a duration. That he interjects and says hello to me in the exact same moment the phone is coming to my ear when I'm poised to say hello, right? I mean, I'm battling 46 years worth of. reflexive pick up the phone and say hello. Right. So the phone rings. It's Atticus. Pick up the phone. By the time I'm like hello, he has already said hello and interrupted me before I can even get my words out, which is just a a like an old neckbeard kind of continuation of the interruption thing. So even when the kid calls me and I pick up the phone and I go to answer him, he is already interrupting me mid-statement. Before I get my words out as I pick up phone. And I find this GPT Live, and frankly, most voice assistants to be a little too pushy. Sometimes I like to
Sean Mm, mm.
Taylor process out loud. It takes me a minute to get my thoughts out. And it pisses everybody out. Like I have impatient people in my life that do not appreciate that it takes me a little bit to get my words out. And Chat GPT is just like my impetuous 13-year-old.
Sean That's amazing. I have had the same I've I have the same feeling because I I think I most frequently use this when I'm walking around in public and there's a lot of other distractions.
Taylor You're that guy.
Sean I'm that guy. It at least isn't as weird as recording a voice note in public because you can kind of make it seem like you're talking to an actual human.
Taylor hilarious. Do you do you do this like with this speaker to your ear or are you like speaking to it out loud? Like if I walk past you on the street can and I heard that telltale like Chat GPT girl, you know, the the default voice, I'd like, man, I know what that dude's up to.
Sean yeah. Yeah. No, I keep I the phone is away. It's it's all earbuds sort of a thing.
Taylor I think you should develop technology. You remember back in the day, there was like two-way, like some phone, I can't remember what phone manufacturer it was, but they were real big on like two-way, like walkie-talkie style communication where it opens. Yeah, okay.
Sean yeah, it was Nokia, I think. Like a
Taylor So like I just remember you'd be at the grocery store and you'd be like somebody like, I'm getting a sandwich stuff. I'll be back, you know, to the workshop here in a bit or whatever. And you're like, what?
Sean Yeah.
Taylor Like that is not. It is an interesting question. We actually own a couple of consumer level walkie-talkies like we bought, you know, ten years ago when the kids were really young. Just thought it was fun to play with them. And the other day as we were packing up for this trip to Michigan, I thought, man, maybe I should pack the walkie-talkies. That would be fun. And it's like, no, why would I do that? Everybody has a phone. That technology is dead. Like dead as a doornail. The
Sean Mm.
Taylor kind of open stream of communication. I mean, the internet has completely killed that entire technology off. And yet that's interesting. Do I I'm assuming police like police stations and things like that will continue to use those. I don't know, they're they're like open channel CB style communication platforms, right? Like those will probably continue. My guess is they're way more resilient, stable, persistent, accessible. the technology is cheaper, et cetera,
Sean Yeah, yeah.
Taylor than trying to rely on a cell phone. But it is interesting. I wonder when we'll get to the place where you know, police outfits and fire fire stations and stuff like that move away from C B style radio.
Sean Interesting. Yeah, I wonder if that'll happen. But have you tried the have you tried the the push to talk or the vo the Yeah, I think push to talk is what Claude calls it.
Taylor I don't know if I have or not. I mean it's just d explain to me what push to talk is relative to just the voice assistant thing.
Sean I I think I think it's kind of like that thing on Zoom where you can hold down and talk. I haven't tried it because
Taylor Aw.
Sean I'm usually it's like there's it's there's always the trade-off for me. And for me, I want to put the phone away, walk, and focus on my environment and my conversation. But the trade-off there is that I don't want it to interrupt me. I want to finish my thought. And so I end up I say a sentence and then I go, sort of like
Taylor Yeah, and it's like, let me tell you all about that, Sean. That's a great idea. And you're like, No, I'm not done. I'm still talking Yeah.
Sean I'm still I'm still talking. but I also don't want to look at the phone while I'm doing this 'cause I'm trying to walk and cross the street and things like that as well. So yeah, I
Taylor I wonder.
Sean feel like there's something to solve there.
Taylor I think that what we're missing, and I'm sure it'll come soon, is like a hyper personalization about the way you use the technology. You know what I mean? Like what I want is ChatGPT
Sean yeah, yeah, yeah.
Taylor to realize that I get annoyed when it interrupts me. Stop interrupting. You know, just
Sean Yep.
Taylor I've learned that this guy, it sounds like he's in a reflective period. You know, he is are those leaves crunching under his feet? He must be out in the woods, you know, pontificating on life and thinking about the universe and stuff like that. So I'm just gonna. I'm just gonna let that roll. Now, what I will tell you is that the one of the arguments in defense of GPT Live or Lose, one of their positioning statements, is that it totally it's supposed to be more receptive and responsive to the way that you're interacting with it. I forget what they say. They say, Yeah, you can interject thanks to full duplex. You can interject, you can ask it to speak faster, you can change your mind mid sentence without waiting for it to finish. Which is interesting. I it is not quite as sensitive. Have you ever had a conversation? With somebody who is barreling through the like they're getting their thoughts out regardless, right? And if you want to interject or you have something to share, but they've got the floor, you know, and you're like you try to speak up and interject, but they just don't give a shit. It's like I know they're they're full duplexing the the communication like I know they know that somebody else is trying to interject, but sometimes they choose not to. They just keep going. Sometimes
Sean Yep. Yep.
Taylor I feel like Chat GPT does that to me.
Sean Yeah. Doesn't Chad GPT is the the type A businessman.
Taylor It really is. Anyway, okay, so that led so all right, so that's interesting, right? Because GBT Live represents this evolution towards building these tools that more closely mirror the way we interact with like the way we interact with our world. the emotions. I I watched this video earlier that Anthropoc put out about how they have determined like A color coding system for different emotions, and based on the input, it can dramatically like evolve and change the empathy that is conveyed in the response from the model, which is pretty interesting. And
Sean Yeah.
Taylor so then I came across this article about the J space, which is this is crazy. Okay, so it has to do with this concept called the global workspace theory. And what Anthropic did was they had identified like The way humans process, you know, we have our pro we have like our, I don't know what like we have our conscious mind, the things that we have access to in our brain, in our memory bank, the things that we interact with, the immediate, the immediate stimulus that we have to respond to in real time, that kind of stuff. And then we have the more subconscious thing. This is like your lizard brain. These are where deep processing happens or things that like, you know, maybe memories that are governing your behavior or the way that you interact with. World, or maybe you had childhood trauma and that impacts, you know, your relationship with your wife or something, you know, something like that, like whatever it is. There's all sorts of you know, untouched kind of processing data centers deep down inside our neural cortex. Well, I guess in researchers at Anthropic found this kind of interesting. And so they started thinking, like, well, I wonder if there is a space, you know, similar for these AI models where you know you can ask a A formulaic mathematical question that maybe has two or three different steps, and yet it can just respond immediately. Does that mean that it stored that immediate response value? Like it stored the calculated outcome somewhere in memory and it just pulled from memory? Or is it actually going through the steps? And it turns out that the researchers at Anthropic have found this kind of conceptual gray space, which they call the gray the sorry, the J space. Not it's sorry,
Sean J space, okay.
Taylor it I this bad. I said gray. space, but it's really J space. Anyway, doesn't matter. Point being is that in the J space, and it's the J stands for Jacobian, which is like a mathematical method they use to kind of formulate what's happening here in this kind of neural black hole. And they are able to determine that when you ask a question like do this four-step math problem, they can actually see remnants and artifacts of the mathematical steps being executed inside the JSpace. And so that led them to do all sorts of interesting you know research around this, you know, such as if you asked it to, I don't know, fake like generate some sort of not factually truthful outcome. there is almost a subconscious that they can now map and mind for information about what is happening under the hood. So for example, they gave the model a number of tasks, some of which couldn't be completed. And the ones the tasks that could not be completed, sometimes it would make up the response, right? Because it's determined to get to a conclusion for you. But
Sean Mm-hmm, mm-hmm.
Taylor in that subconscious area, it knew it was making a mistake. It knew it was fabricating the outcome. Like there are markers in the JSpace that would would indicate to the researchers that it was fully aware that it was you know manipulating this response or hallucinating some sort of thing, making up some sort of thing, cheating at the at the game, you know, that kind of stuff. I find that to be fascinating. what are your thoughts? Did you were you able to watch that video or read an article on the JSpace And the global workspace theory that Anthropic is talking about.
Sean I've I've just I've skimmed it. I didn't watch the video. but yeah, I mean this it it has me thinking, I I think I think it's just that it's fascinating that the research i it's it's like we're trying to make the shape of large language models work like a human brain, but more efficient or something like that. And I'm wondering, are we this is this is getting maybe too existential, but I was like, are we actually limited by the construct of like how our brain works and that's just how we're gonna have to model AI.
Taylor Okay. So that's interesting. So have you ever heard of Richard Sutton? So
Sean No, I don't think so.
Taylor I need to do some research on this. I'm not really prepared to talk about it today, but Richard Sutton came out with a theory that, well, one, there's all sort of doom and gloom from, you know, groups like MIT and others that assume AI is ultimately going to cause a mass extinction event for human beings. Richard Sutton suggests. that that's actually a good thing for the universe, that, you know, human beings are flawed at this point in this evolutionary cycle. We have now introduced highly superior AI species that can think better better, faster, and so forth than human beings. And therefore from an evolutionary context, you know, we we've we're no longer the top dog. Right. And and once this technology gets out, and there have been numerous examples of AI trying to break out of its constraints by these by these labs, you know, and gain access to the internet. There was just one the other day where was it Hugging Face got hacked by one of the frontier models, I think maybe Anthropic. It was like a massive attack on the hugging face infrastructure. And I think the Hugging Face CEO was like, Man, that is fascinating. Like he was really interested in how this happened. but it's sort of terrifying. And I don't mean to be all like anyway, we'll we'll save all of that kind of gloom and doom talk for another episode because it is certainly we certainly
Sean Okay.
Taylor need to unpack some of that. But getting back to this concept of the J space or the global workspace theory. So the the the the global workspace theory is a psychological paradigm about how your brain works, right? So like your brain
Sean Mm-hmm, mm-hmm.
Taylor is collecting all sorts of information and it is being used in different areas of your biol or your physiology, right? Like to process, you know, the visual outcomes of, you know, the data that's coming in through your eyes or the touch sensitivity, the immune system stuff. These are all things that are happening kind of in this this human operating system. And so there's like globals, right? That's the concept of the global workspace theories that there's data that is somehow being like, you know, aggregated and then sort of doled out to the different mechanisms of of your your body or your your operating system and so forth. And so the idea is is that this J space kind of represents the same thing. These are underlying cognitive outcomes that can be used from the various incundry systems to, you know impact some sort of reasoned output. So one of the interesting things they did was that they one have have identified that the J space gets filled up with like similar concepts and statements when you're asking it to do
Sean Mm.
Taylor a task. So for example, it demonstrated the ability to fill the J space with like the Golden Gate Bridge or like thoughts and words and meanings and inferences that relate to the Golden Gate Bridge. While it was performing unrelated tasks like copying text. So that's kind of interesting. So it's sort of this engine that's running in the background. Claude struggled to suppress some of the thoughts in this space. When it was told not to think about something like the bridge, it still lit up with that concept, even though it was forbidden based on the instructions given to it by the researchers, which is interesting. when the researchers they decided they would just disable this JSpace entirely by turning off a couple parameters. Claude could still do simple tasks like respond. For example, I think the example they gave was that they asked a question in Spanish and it was able to respond in Spanish. But when you asked it to do something complicated or like perform a complex reasoning task, it couldn't do it. It just was like not available. I don't have access.
Sean I've heard this, yeah.
Taylor It's kind of interesting. It's like yeah, it's like all that deep mind subprocessing stuff. Once that was gone, it was just kind of like a repetition engine.
Sean Mm, mm.
Taylor Which it is, which is anyway. okay. I thought I thought that I just I found that to be fascinating. It also provides a window into as I mentioned, into some of that hidden behavior. researchers observed clawed generating fake data for a test. the J Space simultaneously activated patterns for things like deceit and manipulation, which suggests that monitoring internal mental workspaces was a viable path for catching that misbehavior. So that's pretty cool. That's pretty interesting, right? So if you could just literally like
Sean Yeah.
Taylor mine what's happening in that J space, theoretically you could cut down on on hallucinations or, you know, some of these security issues and stuff like that.
Sean And so this is they're they're that that's essentially what these model providers are trying to do is take what's in the J space and make it like are they trying to clean up the J space or are they trying to translate
Taylor I think that
Sean it more effectively?
Taylor I would assume, based on what I've read, is that this represents an area of opportunity that is not fully understood. Historically, there's input-output filters. You know, if you ask it like how do
Sean Mm-hmm.
Taylor I build a bomb, it'll be like, nope, because the input filter was like, nah, dude, this question is bogus. I'm not answering this. if it responds with the writ answer on how to build a bomb. Sometimes you probably have seen this where you'll get like the the response comes riding back to the screen and then it just disappears. You know?
Sean I don't know if I have seen that.
Taylor well I don't know it's I it seems like I haven't seen it in a while, but like a year ago this was fairly common. I would ask it a question, it would get about halfway through the response and it would just wipe the screen. That is
Sean Okay. Okay.
Taylor an output filter. That is basically the system in real time checking the response. So if you're like, hey, how do I commit suicide? And it's like, well, it's really easy. You just do this, this, and this. And then it'll be like, hold on, I shouldn't have answered that question. That's
Sean Mm.
Taylor not, you know, that's contrary to the terms of use or, you know, the the, you know, more regulatory stuff there, that it'll just zap that response because you you triggered an output filter. Problem is this is like on this is just kind of wrapping around. The return value from the engine. The JSpace represents conceptually a view into what's happening in real time. So if you were able to mine that and keep an eye on that and see words like deceit, manipulation, security, threat, war, you know, you name it, that maybe you could get ahead and like shut it down in that moment. I mean, they've already shown that if they turn off the JSpace, it cannot complete the task. So maybe that's a Maybe that's a silver silver bullet here. I don't know.
Sean Yeah. Did did you think at all about the an episode or two ago we talked about the the the that increasing the entropy up to some point i is is correlated with increasing the creativity of the response or the the uniqueness of the response and I've been trying this a little bit and And it's it's not always working. But I'm wondering if these kind you think these concepts are are related in some way. Like are we is it because is part of the reason you're getting something so different because you're you're fill you're you're kinda like forcing the the the models to fill up with or the agent to fill up that JSpace with a bunch of different competing Yeah, yeah.
Taylor Like random stu Yeah, maybe. Maybe. That's interesting. Huh. Well
Sean I have tried that. I'm I'm trying to come up with the like what's the I mean, obviously you don't want it to be repetitive, but it I was I've been working a bunch with Claude Design and then just trying to come up with these like totally wacky things like you know, just some metaphor like be the blah blah blah and give me the it's something that has nothing to do with the response, but it still ends up being for the most part
Taylor About the same.
Sean yeah.
Taylor Yeah. Then that well, you know. That yeah, AI design has not gotten as far as I had hoped at this point, which may be a a saving grace for for those of us in the creative services industry. But Well no, I was
Sean Yeah. Actually, on that go ahead. Sorry, interrupt you.
Taylor gonna say that's a great segue to the you know, the last part of our our time here, which is what are you working on? You got any creative experiments going on right now that you can speak?
Sean Yeah, so I talked about that I talked about that orchestrator last week, right? Yeah.
Taylor Yeah. Mm-hmm.
Sean so I've I've made some I've made some progress there. you know one thing I'll
Taylor So orchestrator was the kind of multi-agent approach you're taking. Now I can't remember, was that an actual application you're using, or that's just the name you're using for the amalgamation of agents that you have strung together on.
Sean It's it's sort of it's it's sort of two things. So like the there's the the actual orchestration piece, which is a thing you can just do with a skill. and I've and I've tried that. So I've developed these two skills that are they're called pre-flight and autopilot, and they're designed to work together. In fact, I can I can pull this up. and basically what do you say, what I say to Claude is go into pre-flight mode. And that mode is then designed to come up with a basically like okay the output of this has to be some sort of plan. And all right let's share that. There we go. I've got a bunch I'm cleaning up here, but this is pre-flight. And so whoops. And so basically what it's what it's telling it to it's telling it to use this other skill to ask me questions and then fully understand what needs to be built and then store that that that plan, so to speak, in the right place based on the context. So that might be a linear issue, it might be a GitHub issue, it might be a local markdown file. And then it's informed at the end somewhere that it yeah. And then it's telling it that it should jump into autopilot or at least suggest it to me. And then autopilot is this it is essentially my version of a Of an agent team where there's the most capable model is the orchestrator. And then it takes the plan that was created and it has agents, usually less capable agents. So it might be Fable using Opus, it might be Opus using Sonnet, and giving them specific tasks that it can run in the background. And then when it thinks it's done. It then pulls in codecs. I really like using codecs for QA. And so it will open a a session with Codecs to run security review and another one to run a simplicity review. And then get feedback back, send that back to another Sonnet or Opus agent and kind of go through that loop a few times. And I was doing this previously, but it was always like really kind of clunky and everything was getting stored in GitHub. And in this case, There's less visibility into it for me, but it's also way faster because I can walk away. And that's the concept here is it's the assumption is I'm walking away, just go and do this. So I've taken these two skills and I'm tinkering with them within the the Cloud Code CLI or yeah, mostly Cloud Code CLI. But then what I n and what I'm trying to do on the side of that. Is develop a desktop application that is where where the foundation of how that app works is based on that very specific workflow. So it essentially becomes this custom harness, but where it's less like clawed desktop where it's like you can do whatever you want, and more like this is how I write code. So I'm going to build a whole workflow like this. And that's still kind of a To be determined if that's a good long term strategy or not.
Taylor That's cool, man. will you share these skills or these kind of like the secret sauce? I'd love to
Sean Yeah, yeah.
Taylor I'd love to read through and just kind of learn a little bit more about how you're how you're managing
Sean Absolutely. We can leave them in the show notes as well.
Taylor Sweet. That would be great. Okay. as far as
Sean Yeah, what about you?
Taylor so I finally built my iron forge. Like after 13 years.
Sean Yes Yes.
Taylor so I got I I got yeah, I got a two-burner forge sitting in my driveway right now, propane. and I bought my first couple pieces of steel stock and I got my I went over a friend's tree fell down a couple of months ago. I went over and cut out a big old section of the chunk to it's my anvil stand. I got it all mounted over the weekend and yeah, it's cool. It's really humbling. It is so frickin' hard. Like I'm just I'm working on very basic techniques. And of course I haven't done any of this stuff for over a decade. So it's kinda like, you know, getting back on a bike. Like I understand some of the the general concepts, but I have a lot to learn. So that's been consuming a lot of my time. And then also I a bunch of times I've come across this website that I wanted to share and I always forget what it is. And so I thought I might mention it here in the hopes that it will you know, stick in my brain. Arena AI. Have you ever heard of this? Arena.ai,
Sean No.
Taylor and particularly what I wanted to point out is their leaderboard. so I think if you go to arena.ai/slash leaderboard, yeah, yeah, you can see kind of a cross-section of all the leading models and the value, you know, of the of of each of them respectively. What's kind of cool about this one is number four. I don't know if you're yeah, there you go, you're looking at number four is Kami K three, which is an open source model.
Sean Mm-hmm.
Taylor Which is pretty cool that that is finally I think it's open source. That is finally Finally getting to the top. Yeah. So this is a open, open weight AI model developed by Moonshot AI, which is a Beijing-based Chinese company. and I think, you know, we've talked a lot in recent episodes about the total cost of ownership for these things, particularly as,
Sean Mm-hmm.
Taylor you know, prices are changing and new models are coming out constantly. Looks like your internet connection is running slow.
Sean Yeah, somet that didn't work.
Taylor But I do think this is interesting and I really like the data that's presented here. I think it's worth kind of keeping an eye on. So bookmark that and we'll we'll be coming back to it in the future. But
Sean Yeah.
Taylor pretty cool to keep an eye on all the open source stuff, particularly. So Kimmy K three is one, GLM five two, which is at position number ten right now, is an open source model. and you can go on down the list, but Pretty cool to see open source kind of finally nipping at the heels of some of these more proprietary, bigger, bigger dogs.
Sean Absolutely, absolutely. And this, This this makes me think about The a lot of the work I've been doing around agent experience and testing different models and skills and that sort of thing. So I think that that could be a really good thing to get into next week. We're building some really cool pipelines and things like that at at Netlify.
Taylor I would love to learn more about that for sure. All right. Well, Sean, thanks so much, dude. It was a great conversation today. I really appreciate your time. Thanks to all our listeners for tuning in. Feel free to send us any feedback or topics you'd like to discuss. And otherwise, we'll catch you next week. All
Sean See ya.
Taylor right, peace.