Disclaimer: These are personal learning notes from a ServiceNow architect trying to make sense of what’s available, what works, and what matters.

Source / Inspiration: 👉 https://www.youtube.com/watch?v=BuI8bRuyBjk

As part of my ongoing effort to understand how generative AI fits into ServiceNow’s platform, I spent time looking at the Generative AI Controller. It’s one of those features that sits between the out-of-box Now Assist capabilities and full custom development — and that middle ground is exactly where most real-world customization needs live.

This article captures what I learned: how it works, what’s needed to set it up, a concrete use case built in Flow Designer, and my own reflections on where it falls short or could go further. The original material I studied was recorded around the Vancouver release, but I’ve updated the prerequisites and context to reflect the current state as of Zurich.

What the Generative AI Controller Does

ServiceNow ships several GenAI capabilities as part of Now Assist — case summarization, resolution notes generation, Now Assist search. These are ready-made, tied to ServiceNow’s internal model (Now LLM), and work out of the box for supported use cases.

But what if I want to build my own generative AI workflow? Something a bit different from what’s delivered, or tailored to a specific business process? That’s where the Generative AI Controller comes in. It provides a low-code way to embed generative AI into ServiceNow workflows — Flow Designer, Virtual Agent Designer, or scripting — without dealing directly with REST APIs or Integration Hub plumbing. The integration complexity is abstracted away. I connect to a provider, pick a capability, drag and drop it into a flow.

Prerequisites and Constraints (Updated for Zurich)

Before getting started, there are a few hard requirements. A Now Assist SKU is required — Pro Plus or Enterprise Plus. This means it’s not available on a basic license. The feature was introduced in Vancouver Patch 2, so that’s the minimum version, though Zurich is recommended.

The provider landscape has expanded considerably since the early days. At the Vancouver release, only Azure OpenAI and OpenAI were supported, with Google Palm announced as “coming soon.” As of Zurich, the list includes Azure OpenAI, OpenAI, Google Gemini (the successor to Palm), AWS Claude via Amazon Bedrock, IBM WatsonX, and Aleph Alpha. There’s also a Generic LLM Connector that allows integration with any external LLM provider. That’s a meaningful evolution — from two locked-in providers to a genuinely open model.

One constraint remains unchanged: there is no public access to Now LLM through the controller. ServiceNow uses Now LLM internally for its own Now Assist solutions, but customers cannot route their custom controller flows through it. Only third-party providers are supported.

The most probable reason is cost. Now LLM runs on ServiceNow’s own GPU infrastructure, and GenAI prompts that hit that model consume expensive resources. ServiceNow has said as much in community discussions. Opening Now LLM to arbitrary custom flows via the controller would create unpredictable compute demand they can’t easily forecast or meter. On top of that, there’s a data governance angle: when using Now LLM, data leaves the instance temporarily but stays within ServiceNow’s regional data centres and is deleted after processing. Allowing custom controller access would complicate that tight data residency guarantee. And commercially, Now LLM is the differentiator that justifies the premium Now Assist pricing — keeping it exclusive to managed features makes strategic sense.

Setting Up the Connection

The setup involves three main steps.

First, install the Generative AI Controller store app from the ServiceNow Store. This installs the controller itself along with provider-specific spokes (e.g., the OpenAI spoke, the Azure spoke). These come bundled when installing from a Now Assist SKU.

Second, configure the connection and credentials. Under Connections and Credentials, ServiceNow provides out-of-box credential aliases for each supported provider. For Azure, the connection URL and API key come from the Azure portal — specifically from the Keys and Endpoints section of the Azure OpenAI resource. Copy-paste the endpoint as the connection URL, copy-paste one of the keys as the API key, click create. That’s it for the connection.

Third, set the default generative AI provider in System Properties. Navigate to the generative AI properties and specify the provider name as the default.

Activating Capabilities

Once the connection is established, the next step is to activate the specific capabilities to use. These are managed in the sys_one_extend_capability table. Out of the box, several capabilities are provided: Generate Content (for article generation based on input), Generic Prompt (open-ended — ask whatever I want), Sentiment Analysis (returns negative, neutral, or positive), and Summarize (takes input text and produces a summary). The Zurich release also added capabilities like Summarize Incident and Code Assist.

Each capability has configuration records that determine which model to use. The options depend on the provider — for Azure, the options include Azure OpenAI chat completion and Azure GPT-3.5 LLM; for OpenAI, GPT-4 and GPT-3.5 variants. I select the one I want as the default for that capability, ensure the active flag is true, and move on to building.

A Note on the Zurich Architectural Shift

One development worth flagging: in Zurich, the ability to create custom generative AI applications within Virtual Agent Designer and via scripts has been removed. ServiceNow now steers custom GenAI work toward the Now Assist Skill Kit instead. The Generative AI Controller still works in Flow Designer, but the direction is clear — ServiceNow is moving away from open-ended custom scripting and toward managed, governed AI solutions. For architects planning custom GenAI implementations, this is a shift worth tracking.

Building a Use Case: Summarize Work Notes on Incident Resolution

The use case I explored is practical: when an incident is resolved, automatically summarize all its work notes and write that summary into the resolution notes field.

Now Assist doesn’t ship this exact use case out of the box, but it’s a reasonable scenario. Work notes accumulate throughout the life of an incident — troubleshooting steps, vendor interactions, escalation decisions. When an agent finally resolves the ticket, the resolution notes should capture what happened. In practice, agents often write something minimal or vague. Automating a summary from the work notes could improve that.

Here’s how it’s built in Flow Designer.

The trigger: a record update on the Incident table, with a condition that the state changes to “Resolved.” Run once.

The challenge with journal fields: the Summarize action expects a string input. For a regular string field like short description, I’d just use the data pill picker to grab the field value directly. But work notes is a journal field, not a simple string. Passing it straight into the summarize action won’t work — the action expects text. The workaround is a script step to convert the journal field to a string using .getDisplayValue(). Anyone who’s worked with GlideRecord will recognize the pattern.

The summarize action: drag in the Summarize action from the Generative AI Controller spoke. Feed it the output of the script step (the stringified work notes). The action sends the text to the configured AI provider and returns a response.

The update step: an Update Record action on the same incident, writing the summarize action’s response into the resolution notes field.

Save. Test. The flow executes, the summarization appears in the resolution notes field. Done.

From there, the flow could be attached to a UI action to create a manual “Summarize” button, or left to run automatically on state change. The same pattern applies to any of the other controller capabilities — sentiment analysis would return a sentiment value in the response field instead of a summary.

Discussion: Where It Gets Interesting — and Where It Doesn’t

The low-code aspect is real. No REST API calls to study, no integration hub action to configure from scratch. The connection setup is straightforward, the spoke actions are drag-and-drop. For a platform that prides itself on extensibility, this is a clean implementation.

But the more I think about this specific use case, the more questions I have.

The quality-in, quality-out problem. Work notes in most organisations I’ve worked with are inconsistent at best. Agents write fragments, paste error logs without context, or add one-liners like “called the user, no answer.” If the work notes are poor, the summary will be poor. Generative AI doesn’t add information that isn’t there — it reformulates what exists. So automating a summary of bad notes just produces a well-formatted bad summary. The real problem isn’t summarization; it’s the quality of what agents capture during incident handling.

Structured output. The summarize capability returns plain text. But in many organisations, resolution notes have a purpose beyond closing the ticket. They feed into knowledge articles, they’re referenced in post-incident reviews, they serve as precedent for similar future incidents. A free-text summary may not be enough. What I’d want is structured formatting — something crisp and reusable. Sections like “Root Cause,” “Steps Taken,” “Resolution,” “Recommendations.” The Generic Prompt capability could potentially handle this with the right prompt engineering, but it would take some iteration to get reliable, consistent formatting.

Enrichment from similar incidents. What if the summarization didn’t just condense the current incident’s work notes, but also pulled relevant information from similar past incidents that had the same resolution? Incidents are often resolved the same way repeatedly, but the depth of documentation varies. One incident might have detailed steps while another has a one-liner. Enriching the resolution notes with context from better-documented similar cases would produce richer, more useful output. This would require combining the controller with Predictive Intelligence’s similarity solutions or a custom lookup — possible, but not out of the box.

Voice capture. People don’t like to write. That’s a universal truth in ITSM. They prefer to speak, especially during high-pressure incidents. What if work notes could be captured via voice recording and then transcribed and summarized? The summarization part is already handled by the controller. The missing piece is speech-to-text capture within the ServiceNow agent workspace. If that input channel existed, the quality of work notes would likely improve because agents would actually use it.

Linking to knowledge and Problem. Once I have a good resolution summary, the next logical step is to match it against existing knowledge base articles or Known errors. Does an article already describe this resolution? If so, relate the incident to it. If not, flag it as a candidate for KB creation. This closes the loop between incident management and knowledge management — something most organisations aspire to but rarely achieve systematically.

Self-service and automation potential. Finally, and this is more strategic: once resolution notes are structured and enriched, they become a data source for deeper analysis. Could this resolution have been automated via an interface? Could the end user have self-helped with the right KB article? These are the questions that drive shift-left and self-service strategies, and the data to answer them often lives in resolution notes — if those notes are good enough.

Closing Thoughts

The Generative AI Controller is a practical addition to ServiceNow’s AI toolkit. It fills a real gap between the fixed out-of-box Now Assist capabilities and full custom API integration. The setup is fast, the low-code approach works, and for straightforward use cases like text summarization or sentiment analysis, it delivers. The expanding provider list — from two options at Vancouver to a genuinely multi-cloud model at Zurich — makes it more viable for organisations with existing cloud commitments.

Where it gets more challenging is in the quality of inputs and the sophistication of outputs. The tool does what it’s told, but the value depends entirely on the data it receives and how well I prompt it. That’s not a criticism of the controller — it’s a reminder that tooling is only one part of the equation. Process maturity, agent discipline, and data quality are the other parts, and no amount of GenAI abstracts those away.

And with the Zurich shift toward Now Assist Skill Kit and away from open-ended scripting, the strategic question for architects becomes: how much of my GenAI customization do I build through the controller versus adopting the managed Skill Kit path? That tension between flexibility and governance is probably the most interesting design decision in this space right now.

I don’t claim to have the full picture. These are notes from someone learning in public, and I welcome corrections, different perspectives, or better ideas.