the new user persona

empathy is generally called out as one of the most important skills in a product manager. it's true: the voice of the customer should drive decision-making. ever since i started my career, my framework has been pretty simple. i talk to customers, understand what they're trying to do, and build an intuition for their world, goals, and pains. i then use that intuition to prioritize and build. i wouldn't call myself "data-driven" in the sense most people mean. for me, numbers are just a signpost that can guide me toward a conversation i can have with a customer, with the goal of building intuition.

when i say "customer", i don't necessarily mean the end user. i've been leading our AI platform charter where the "customer" is either a developer building a system and exploring our service through some playground-type experience or API docs, or a consumer application using our service via an API interface. in the context of the latter, though, i have a strong intuition for the end-user who will be using the application. i've shipped things for lawyers and legal ops people, and i've had countless conversations with them, so i have a feel for them. which is to say that even though the platform is directly consumed only internally, by internal "customers", the end outcome is not lost on me.

then there are agents

i think most people still treat agents as some abstract, non-alive users that work kind of like API consumers. we build and ship some API docs (and give it a fancy name: MCP). it's easy to forget that these consumers can actually think and make autonomous decisions. exposing a toolkit and expecting the agent to figure it out is like putting a novice user into Jira and expecting them to figure it out. given enough time and feedback, they will, but it's a frustrating experience.

the reason we let it slide with agents is because agents cannot feel frustration. "throw things at the LLM and let it figure it out" works only because the LLM doesn't complain (okay--sometimes it does).

source: reddit

most times, after enough trial and error, a good agent figures it out. but it's like the old SaaS way of exposing every single feature with buttons all over the place. users hated old SaaS. this is why Jira gets so much hate.

since agents aren't usually frustrated in the human sense, the negative effect manifests in the end user instead. it takes too long, consumes too many tokens, leads to wasted tool calls, and in general lends itself to a poor user perception of your product.

skills plugs the gap in one way

i like to think of skills as a help center article, or a product guide. if you've used a terrible product, you know the ones: those articles, either created by some ops person internally or shipped with the product itself, with screenshots all over the place that you have to follow step by step. a skill is the agent equivalent of that. sure, it works. but is it the best experience? what if the agent interface changes? you'd have to update a well-written skill every time.

think about it this way. if your product costs the agent less time and tokens, you'll come out on top as the system of record of choice. that's the only moat that will matter as a SaaS company: whether the knowledge in your SoR is helping my agent help me achieve my outcomes. the answer to fixing problems cannot be writing a good skill. that's like saying your product works because it has good help center documentation. it's a band-aid on a poorly executed product interface, just not for humans.

empathize with the agent

i have really started empathizing with the agent persona. i have seen enough failure traces to know what an agent does when it's interacting with my product, and where it goes wrong. before 2022, Amplitude and Mixpanel dashboards should have been your DAU products. in 2026, you need to live inside wherever you're logging traces. Langsmith for me. every tool call attempt is a click a user would have made. every failed tool call is a rage click. hallucination from context bloat caused by a noisy tool result is equivalent to a user getting confused.

think of an agent like a specialist acting on behalf of a human user. how would the user do it themselves? maybe there's a more efficient way for the agent to do it. when you sit down with a trace, the questions you should be asking are basic:

• what information and constraints does the agent have
• what's the agent's JTBD
• what will help it get to the right information
• how do you give the agent meta information so it can find the right information
• what is the agent trying to do when it calls a tool

these are the same questions you'd ask of any user. the only difference is that the agent can't tell you the answers directly. you have to infer them from traces. or, better, you have to design the system so the agent tells you.

just like a user should see progressive disclosure of information, so should an agent. over a year of building an agent has taught me one thing well: don't bombard it with information. wherever possible, give it helpful cues. tool call responses should have good feedback. "here's what you did, here's what i thought you wanted, here's what i've given you, here's how you can possibly use it, here's some suggested next steps." humans get all of this in a good human interface. agents should get it in a good agent interface. some people call this context engineering, but really it's just common sense.

the same applies to tool responses themselves. cleanup matters. no long nested JSON responses with half the keys missing or empty. structured machine API calls required structured outputs because the consumer was a deterministic program. that's not true anymore. LLMs can extrapolate pretty well, but the cost of feeding them noise shows up as context bloat and confused trajectories. clean up your responses.

helpful error messages are part of the same idea. avoid generic "oops, this failed." no verbose stack traces dumped into context. aim for: "you tried this, this is why it didn't work, here's what you should try next."

an example: i had an instance recently where a one-line change made a flow that was failing start working. the agent had access to older tool shapes in its prior context, and we'd added a validation that meant it couldn't reuse the same skills from context. it had to reload them. it didn't know that. one hint in the error message:

Hint: To reuse a tool you've already used before, you have to unlock it by finding the relevant skill again.

and it worked. one line. that's the kind of leverage you don't get with traditional product copy.

ux ax research at scale

wouldn't it be great if humans could tell you what they wanted to do while they were doing it? it would make it so much easier to build good human interfaces. UX research 101 is telling the user to say their intents out loud while they're doing something.

you can get this for free with a toolkit, at scale. along with every tool call, just ask the agent to send you an optional rationale. sweeten the pot for the agent. as a toolkit owner, you know what the best tools are mapped to the best intent. own it, and respond with a hint every time the agent tells you what it wants to do. it's an unexplored gold mine.

feedback buttons

you collect feedback from humans after a UX session. why not from an agent? share a tool with the agent after they complete an activity. did an action? add to the tool call response: "you are encouraged to provide feedback on using this tool by hitting the feedback tool." build a standard for your own toolkit. collect ASAT (agent satisfaction) scores.

tools are not api wrappers

following from my predictability quadrant, throwing agents at a problem is top-right. we want to move to bottom-left. convert common tool trajectories into packaged tools. package tools by use case. do multiple levels of tool discovery. not everything needs to be visible all at once.

come up with patterns where tools need to be unlocked sequentially for things to work. if those sequences are frequent and agents struggle to execute them well, turn those steps into deterministic workflows inside packaged tool calls. push and pull the right data. ask follow-ups in tool call responses. nudge the agent. don't let it just "figure it out."

accessibility, for agents

accessibility doesn't just help users who require accommodation. in many cases, it makes product usage easier for everyone.

apply the same thinking here. helpful guardrails help novice agents or agents with constraints. say there's an agent that doesn't have code execution, so it can't write code. help it. even the coding agent becomes better for it, because it doesn't have to write code to achieve something you've offered out of the box. build for the weakest model so you capture the largest market. the strongest ones will love you and pick you by default.

the pm job has changed

a PM's job has changed a lot. there's a whole other post to write about prioritizing the right problems. but even within AX there's a lot to do. the best system of record is likely the one where information is easiest to fetch. don't let poor AX be the reason you lose. start early so when it becomes table-stakes, you're the market leader. this is where things like feedback and rationale collection really help.

at the end of the day, an agent is operating on behalf of humans trying to get work done. by doing the right things faster and more reliably, you will help someone achieve a real-world outcome. in marketing-speak, imagine if you could say "uses 3x fewer tokens than competitors." that's a real, measurable gain, for example.