Author: i-genie

Automated smoke testing using Amazon Nova Act headless mode helps development teams validate core functionality in continuous integration and continuous delivery (CI/CD) pipelines. Development teams often deploy code several times daily, so fast testing helps maintain application quality. Traditional end-to-end testing can take hours to complete, creating delays in your CI/CD pipeline.
Smoke testing is a subset of testing that validates the most critical functions of an application work correctly after deployment. These tests focus on key workflows like user login, core navigation, and key transactions rather than exhaustive feature coverage. Smoke tests typically complete in minutes rather than hours, making them ideal for CI/CD pipelines where fast feedback on code changes is essential.
Amazon Nova Act uses AI-powered UI understanding and natural language processing to interact with web applications, replacing traditional CSS selectors. Instead of maintaining brittle CSS selectors and complex test scripts, you can write tests using simple English commands that adapt to UI changes.
This post shows how to implement automated smoke testing using Amazon Nova Act headless mode in CI/CD pipelines. We use SauceDemo, a sample ecommerce application, as our target for demonstration. We demonstrate setting up Amazon Nova Act for headless browser automation in CI/CD environments and creating smoke tests that validate key user workflows. We then show how to implement parallel execution to maximize testing efficiency, configure GitLab CI/CD for automatic test execution on every deployment, and apply best practices for maintainable and scalable test automation.
Solution overview
The solution includes a Python test runner that executes smoke tests, ecommerce workflow validation for complete user journeys, GitLab CI/CD integration for automation, and parallel execution to speed up testing. Headless mode runs browser tests in the background without opening a browser window, which works well for automated testing.
The following diagram illustrates the testing workflow.

We walk through the following steps to implement automated smoke testing with Amazon Nova Act:

Set up your project and dependencies.
Create a smoke test with login validation.
Configure validation for the entire ecommerce workflow.
Configure the automated testing pipeline.
Configure parallel execution.

Prerequisites
To complete this walkthrough, you must have the following:

Access to Amazon Nova Act with API key.
A GitLab repository.
UV package manager. For instructions, refer to Installing uv.
Familiarity with Python and GitLab CI/CD.

Set up project and dependencies
Create your project and install dependencies:

# Create and navigate to project
uv init nova-act-smoke-tests
# Open in VS Code
code nova-act-smoke-tests
# Install required packages
uv add nova-act

UV is a fast Python package manager that handles dependency installation and virtual environment management automatically, similar to npm for Node.js projects.

Create a test runner
Create smoke_tests.py:

import os
from nova_act import NovaAct

# Check API key
if not os.getenv(“NOVA_ACT_API_KEY”):
exit(“❌ Set NOVA_ACT_API_KEY environment variable”)
SAUCEDEMO_URL = “https://www.saucedemo.com/”
with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
nova.act(“Verify you are in the login page”)

print(“✅ Foundation setup complete!”)

Test your setup

Test your setup with the following commands:

export NOVA_ACT_API_KEY=”your-api-key”
uv run smoke_tests.py

Environment variables like NOVA_ACT_API_KEY keep sensitive information secure and separate from your code.
This solution implements the following security features:

Stores API keys in environment variables or .env files (add .env to .gitignore)
Uses different API keys for development, staging, and production environments
Implements key rotation every 90 days using automated scripts or calendar reminders
Monitors API key usage through logs to detect unauthorized access

You now have a modern Python project with Amazon Nova Act configured and ready for testing. Next, we show how to create a working smoke test that uses natural language browser automation.
Create smoke test for login validation
Let’s expand your foundation code to include a complete login test with proper structure.
Add main function and login test
Update smoke_tests.py:

import os
from nova_act import NovaAct

SAUCEDEMO_URL = “https://www.saucedemo.com/”

def test_login_flow():
“””Test complete login flow and product page verification”””
with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
nova.act(“Enter ‘standard_user’ in the username field”)
nova.act(“Enter ‘secret_sauce’ in the password field”)
nova.act(“Click the login button”)
nova.act(“Verify Products appear on the page”)

def main():
# Check API key
if not os.getenv(“NOVA_ACT_API_KEY”):
exit(“❌ Set NOVA_ACT_API_KEY environment variable”)

print(“🚀 Starting Nova Act Smoke Test”)

try:
test_login_flow()
print(“✅ Login test: PASS”)
except Exception as e:
print(f”❌ Login test: FAIL – {e}”)
exit(1)

print(“🎉 All tests passed!”)

if __name__ == “__main__”:
main()

Test your login flow
Run your complete login test:

export NOVA_ACT_API_KEY=”your-api-key”
uv run smoke_tests.py

You should see the following output:

🚀 Starting NovaAct Smoke Test
✅ Login test: PASS
🎉 All tests passed!

Your smoke test now validates a complete user journey that uses natural language with Amazon Nova Act. The test handles page verification to confirm you’re on the login page, form interactions that enter user name and password credentials, action execution that clicks the login button, and success validation that verifies the products page loads correctly. The built-in error handling provides retry logic if the login process encounters any issues, showing how the AI-powered automation of Amazon Nova Act adapts to dynamic web applications without the brittleness of traditional CSS selector-based testing frameworks.
Although a login test provides valuable validation, real-world applications require testing complete user workflows that span multiple pages and complex interactions. Next, we expand the testing capabilities by building a comprehensive ecommerce journey that validates the entire customer experience.
Configure ecommerce workflow validation
Let’s build a comprehensive ecommerce workflow that tests the end-to-end customer journey from login to logout.
Add complete ecommerce test
Update smoke_tests.py to include the full workflow:

import os
from nova_act import NovaAct

SAUCEDEMO_URL = “https://www.saucedemo.com/”

def test_login_flow():
“””Test complete login flow and product page verification”””
with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
nova.act(“Enter ‘standard_user’ in the username field”)
nova.act(“Enter ‘secret_sauce’ in the password field”)
nova.act(“Click the login button”)
nova.act(“Verify Products appear on the page”)

def test_ecommerce_workflow():
“””Test complete e-commerce workflow: login → shop → checkout → logout”””
with NovaAct(starting_page=SAUCEDEMO_URL) as nova:
# Login
nova.act(“Enter ‘standard_user’ in the username field”)
nova.act(“Enter ‘secret_sauce’ in the password field”)
nova.act(“Click the login button”)
nova.act(“Verify Products appear on the page”)

# Shopping
nova.act(“Select Sauce Labs Backpack”)
nova.act(“Add Sauce Labs Backpack to the cart”)
nova.act(“Navigate back to products page”)
nova.act(“Select Sauce Labs Onesie”)
nova.act(“Add Sauce Labs Onesie to the cart”)
nova.act(“Navigate back to products page”)

# Cart verification
nova.act(“Click cart and Navigate to the cart page”)
nova.act(“Verify 2 items are in the cart”)

# Checkout process
nova.act(“Click the Checkout button”)
nova.act(“Enter ‘John’ in the First Name field”)
nova.act(“Enter ‘Doe’ in the Last Name field”)
nova.act(“Enter ‘12345’ in the Zip/Postal Code field”)
nova.act(“Click the Continue button”)

# Order completion
nova.act(“Verify Checkout:Overview page appears”)
nova.act(“Click the Finish button”)
nova.act(“Verify ‘THANK YOU FOR YOUR ORDER’ appears on the page”)

# Return and logout
nova.act(“Click the Back Home button”)
nova.act(“Click the hamburger menu on the left”)
nova.act(“Click the Logout link”)
nova.act(“Verify the user is on the login page”)
def main():
# Check API key
if not os.getenv(“NOVA_ACT_API_KEY”):
exit(“❌ Set NOVA_ACT_API_KEY environment variable”)

print(“🚀 Starting Nova Act E-commerce Tests”)

tests = [
(“Login Flow”, test_login_flow),
(“E-commerce Workflow”, test_ecommerce_workflow)
]

passed = 0
for test_name, test_func in tests:
try:
test_func()
print(f”✅ {test_name}: PASS”)
passed += 1
except Exception as e:
print(f”❌ {test_name}: FAIL – {e}”)

print(f”n📊 Results: {passed}/{len(tests)} tests passed”)

if passed == len(tests):
print(“🎉 All tests passed!”)
else:
exit(1)

if __name__ == “__main__”:
main()

Test your ecommerce workflow
Run your comprehensive test suite:

export NOVA_ACT_API_KEY=”your-api-key”
uv run smoke_tests.py

You should see the following output:

🚀 Starting Nova Act E-commerce Tests
✅ Login Flow: PASS
✅ E-commerce Workflow: PASS
📊 Results: 2/2 tests passed
🎉 All tests passed!

Understanding the ecommerce journey
The workflow tests a complete customer experience:

Authentication – Login with valid credentials
Product discovery – Browse and select products
Shopping cart – Add items and verify cart contents
Checkout process – Enter shipping information
Order completion – Complete purchase and verify success
Navigation – Return to products and log out

The following screenshot shows the step-by-step visual guide of the user journey.

Your smoke tests now validate complete user journeys that mirror real customer experiences. The ecommerce workflow shows how Amazon Nova Act handles complex, multi-step processes across multiple pages. By testing the entire customer journey from authentication through order completion, you’re validating the primary revenue-generating workflows in your application.
This approach reduces maintenance overhead while providing comprehensive coverage of your application’s core functionality.
Running these tests manually provides immediate value, but the real power comes from integrating them into your development workflow. Automating test execution makes sure code changes are validated against your critical user journeys before reaching production.
Configure automated testing pipeline
With your comprehensive ecommerce workflow in place, you’re ready to integrate these tests into your CI pipeline. This step shows how to configure GitLab CI/CD to automatically run these smoke tests on every code change, making sure key user journeys remain functional throughout your development cycle. We show how to configure headless mode for CI environments while maintaining the visual debugging capabilities for local development.
Add headless mode for CI/CD
Update smoke_tests.py to support headless mode for CI environments by adding the following lines to both test functions:

def test_login_flow():
“””Test complete login flow and product page verification”””
headless = os.getenv(“HEADLESS”, “false”).lower() == “true”

with NovaAct(starting_page=SAUCEDEMO_URL, headless=headless) as nova:
# … rest of your test code remains the same

def test_ecommerce_workflow():
“””Test complete e-commerce workflow: login → shop → checkout → logout”””
headless = os.getenv(“HEADLESS”, “false”).lower() == “true”

with NovaAct(starting_page=SAUCEDEMO_URL, headless=headless) as nova:
# … rest of your test code remains the same

Create GitHub Actions workflow
GitLab CI/CD is GitLab’s built-in CI system that automatically runs pipelines when code changes occur. Pipelines are defined in YAML files that specify when to run tests and what steps to execute.
Create .gitlab-ci.yml:

stages:
– test

smoke-tests:
stage: test
image: mcr.microsoft.com/playwright/python:v1.40.0-jammy
rules:
– if: $CI_COMMIT_BRANCH == “main”
– if: $CI_COMMIT_BRANCH == “develop”
– if: $CI_PIPELINE_SOURCE == “merge_request_event”
– if: $CI_PIPELINE_SOURCE == “web”
before_script:
– pip install uv
– uv sync
– uv run playwright install chromium
script:
– uv run python smoke_tests.py
variables:
HEADLESS: ‘true’
NOVA_ACT_SKIP_PLAYWRIGHT_INSTALL: ‘true’

Configure GitLab CI/CD variables
GitLab CI/CD variables provide secure storage for sensitive information like API keys. These values are encrypted and only accessible to your GitLab CI/CD pipelines. Complete the following steps to add a variable:

In your project, choose Settings, CI/CD, and Variables.
Choose Add variable.
For the key, enter NOVA_ACT_API_KEY.
For the value, enter your Amazon Nova Act API key.
Select Mask variable to hide the value in job logs.
Choose Add variable.

Understanding the code changes
The key change is the headless mode configuration:

headless = os.getenv(“HEADLESS”, “false”).lower() == “true”
with NovaAct(starting_page=SAUCEDEMO_URL, headless=headless) as nova:

This configuration provides flexibility for different development environments. During local development when the HEADLESS environment variable is not set, the headless parameter defaults to False, which opens a browser window so you can see the automation in action. This visual feedback is invaluable for debugging test failures and understanding how Amazon Nova Act interacts with your application. In CI/CD environments where HEADLESS is set to true, the browser runs in the background without opening any windows, making it ideal for automated testing pipelines that don’t have display capabilities and need to run efficiently without visual overhead.
Test your CI/CD setup
Push your code to trigger the workflow:

git add .
git commit -m “Add Nova Act smoke tests with CI/CD”
git push origin main

Check the Pipelines section in your GitLab project to see the tests running.

Your smoke tests now run automatically as part of your CI pipeline, providing immediate feedback on code changes. The GitLab CI/CD integration makes sure critical user journeys are validated before any deployment reaches production, reducing the risk of shipping broken functionality to customers.
The implementation shows how modern package management with UV reduces CI/CD pipeline execution time compared to traditional pip installations. Combined with secure API key management through GitLab CI/CD variables, your testing infrastructure follows enterprise security best practices.
As your test suite grows, you might notice that running tests sequentially can become a bottleneck in your deployment pipeline. The next section addresses this challenge by implementing parallel execution to maximize your CI/CD efficiency.
Configure parallel execution
With your CI/CD pipeline successfully validating individual test cases, the next optimization focuses on performance enhancement through parallel execution. Concurrent test execution can reduce your total testing time by running multiple browser instances simultaneously, maximizing the efficiency of your CI/CD resources while maintaining test reliability and isolation.
Add parallel execution framework
Update smoke_tests.py to support concurrent testing:

import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from nova_act import NovaAct

SAUCEDEMO_URL = “https://www.saucedemo.com/”
headless = os.getenv(“HEADLESS”, “false”).lower() == “true”

def test_login_flow():
“””Test complete login flow and product page verification”””

with NovaAct(starting_page=SAUCEDEMO_URL, headless=headless) as nova:
nova.act(“Enter ‘standard_user’ in the username field”)
nova.act(“Enter ‘secret_sauce’ in the password field”)
nova.act(“Click the login button”)
# nova.act(“In case of error, make sure the username and password are correct, if required re-enter the username and password”)
nova.act(“Verify Products appear on the page”)

def test_ecommerce_workflow():
“””Test complete e-commerce workflow: login → shop → checkout → logout”””
with NovaAct(starting_page=SAUCEDEMO_URL, headless=headless) as nova:
# Login
nova.act(“Enter ‘standard_user’ in the username field”)
nova.act(“Enter ‘secret_sauce’ in the password field”)
nova.act(“Click the login button”)
nova.act(“Verify Products appear on the page”)

# Shopping
nova.act(“Select Sauce Labs Backpack”)
nova.act(“Add Sauce Labs Backpack to the cart”)
nova.act(“Navigate back to products page”)
nova.act(“Select Sauce Labs Onesie”)
nova.act(“Add Sauce Labs Onesie to the cart”)
nova.act(“Navigate back to products page”)

# Cart verification
nova.act(“Click cart and Navigate to the cart page”)
nova.act(“Verify 2 items are in the cart”)

# Checkout process
nova.act(“Click the Checkout button”)
nova.act(“Enter ‘John’ in the First Name field”)
nova.act(“Enter ‘Doe’ in the Last Name field”)
nova.act(“Enter ‘12345’ in the Zip/Postal Code field”)
nova.act(“Click the Continue button”)

# Order completion
nova.act(“Verify Checkout:Overview page appears”)
nova.act(“Click the Finish button”)
nova.act(“Verify ‘THANK YOU FOR YOUR ORDER’ appears on the page”)

# Return and logout
nova.act(“Click the Back Home button”)
nova.act(“Click the hamburger menu on the left”)
nova.act(“Click the Logout link”)
nova.act(“Verify the user is on the login page”)

def run_test(test_name, test_func):
“””Execute a single test and return result”””
try:
test_func()
print(f”✅ {test_name}: PASS”)
return True
except Exception as e:
print(f”❌ {test_name}: FAIL – {e}”)
return False

def main():
# Check API key
if not os.getenv(“NOVA_ACT_API_KEY”):
exit(“❌ Set NOVA_ACT_API_KEY environment variable”)

print(“🚀 Starting Nova Act Tests (Parallel)”)

tests = [
(“Login Flow”, test_login_flow),
(“E-commerce Workflow”, test_ecommerce_workflow)
]

# Configure parallel execution
max_workers = int(os.getenv(“MAX_WORKERS”, “2”))

# Run tests in parallel
results = []
with ThreadPoolExecutor(max_workers=max_workers) as executor:
future_to_test = {
executor.submit(run_test, name, func): name
for name, func in tests
}

for future in as_completed(future_to_test):
results.append(future.result())

# Report results
passed = sum(results)
total = len(results)

print(f”n📊 Results: {passed}/{total} tests passed”)

if passed == total:
print(“🎉 All tests passed!”)
else:
exit(1)

if __name__ == “__main__”:
main()

Update GitLab CI/CD for parallel execution
The parallel execution is already configured in your .gitlab-ci.yml with the MAX_WORKERS= “2” variable. The pipeline automatically uses the parallel framework when running the smoke tests.
Test parallel execution
Run your optimized tests:

export NOVA_ACT_API_KEY=”your-api-key”
export MAX_WORKERS=”2″
uv run smoke_tests.py

You should see both tests running simultaneously:

🚀 Starting Nova Act Tests (Parallel)
✅ Login Flow: PASS
✅ E-commerce Workflow: PASS
📊 Results: 2/2 tests passed
🎉 All tests passed!

Understanding parallel execution
ThreadPoolExecutor is a Python class that manages a pool of worker threads, allowing multiple tasks to run simultaneously. In this case, each thread runs a separate browser test, reducing total execution time.

# Configure worker count
max_workers = int(os.getenv(“MAX_WORKERS”, “2”))

# Execute tests concurrently
with ThreadPoolExecutor(max_workers=max_workers) as executor:
future_to_test = {
executor.submit(run_test, name, func): name
for name, func in tests
}

Parallel execution provides benefits such as faster execution (because tests run simultaneously instead of sequentially), configurable workers that adjust based on system resources, resource efficiency that optimizes CI/CD compute time, and scalability that makes it straightforward to add more tests without increasing total runtime.
However, there are important considerations to keep in mind. Each test opens a browser instance (which increases resource usage), tests must be independent of each other to maintain proper isolation, and you must balance worker counts with available CPU and memory limits in CI environments.
Each parallel test uses system resources and incurs API usage. Start with two workers and adjust based on your environment’s capacity and cost requirements. Monitor your Amazon Nova Act usage to optimize the balance between test speed and expenses.
The performance improvement is significant when comparing sequential vs. parallel execution. In sequential execution, tests run one after another with the total time being the sum of all individual test durations. With parallel execution, multiple tests run simultaneously, completing in approximately the time of the longest test, resulting in substantial time savings that become more valuable as your test suite grows.
Your smoke tests now feature concurrent execution that significantly reduces total testing time while maintaining complete test isolation and reliability. The ThreadPoolExecutor implementation allows multiple browser instances to run simultaneously, transforming your sequential test suite into a parallel execution that completes much faster. This performance improvement becomes increasingly valuable as your test suite grows, so comprehensive validation doesn’t become a bottleneck in your deployment pipeline.
The configurable worker count through the MAX_WORKERS environment variable provides flexibility to optimize performance based on available system resources. In CI/CD environments, this allows you to balance test execution speed with resource constraints, and local development can use full system capabilities for faster feedback cycles. The architecture maintains complete test independence, making sure parallel execution doesn’t introduce flakiness or cross-test dependencies that could compromise reliability. As a best practice, keep tests independent—each test should work correctly regardless of execution order or other tests running simultaneously.
Best practices
With your performance-optimized testing framework complete, consider the following practices for production readiness:

Keep tests independent. Tests are not impacted by execution order or other tests running simultaneously.
Add retry logic by wrapping your test functions in try-catch blocks with a retry mechanism for handling transient network issues.
Configure your GitLab CI/CD pipeline with a reasonable timeout and consider adding a scheduled run for daily validation of your production environment.
For ongoing maintenance, establish a rotation schedule for your Amazon Nova Act API keys and monitor your test execution times to catch performance regressions early. As your application grows, you can add new test functions to the parallel execution framework without impacting overall runtime, making this solution highly scalable for future needs.

Clean up
To avoid incurring future charges and maintain security, clean up the resources you created:

Remove or disable unused GitLab CI/CD pipelines
Rotate API keys every 90 days and revoke unused keys.
Delete the repositories provided with this post.
Remove API keys from inactive projects.
Clear cached credentials and temporary files from your local environment.

Conclusion
In this post, we showed how to implement automated smoke testing using Amazon Nova Act headless mode for CI/CD pipelines. We demonstrated how to create comprehensive ecommerce workflow tests that validate user journeys, implement parallel execution for faster test completion, and integrate automated testing with GitLab CI/CD for continuous validation.
The natural language approach using Amazon Nova Act needs less maintenance than traditional frameworks that use CSS selectors. Combined with modern tooling like UV package management and GitLab CI/CD, this solution provides fast, reliable test execution that scales with your development workflow. Your implementation now catches issues before they reach production, providing the fast feedback essential for confident continuous deployment while maintaining high application quality standards.
To learn more about browser automation and testing strategies on AWS, explore the following resources:

Getting Started Using the Nova Act Dev Tools and explore the capability in Nova Act playground
Amazon CloudWatch Synthetics for additional monitoring capabilities
GitLab CI/CD documentation for CI/CD best practices
AWS security best practices for securing your automation infrastructure

Try implementing these smoke tests in your own applications and consider extending the framework with additional test scenarios that match your specific user journeys. Share your experience and any optimizations you discover in the comments section.

About the authors
Sakthi Chellapparimanam Sakthivel is a Solutions Architect at AWS, specializing in .NET modernization and enterprise cloud transformations. He helps GSI and software/services customers build scalable, innovative solutions on AWS. He architects intelligent automation frameworks and GenAI-powered applications that drive measurable business outcomes across diverse industries. Beyond his technical pursuits, Sakthivel enjoys spending quality time with his family and playing cricket.
Shyam Soundar is a Solutions Architect at AWS with an extensive background in security, cost-optimization, and analytics offerings. Shyam works with enterprise customers to help them build and scale applications to achieve their business outcomes with lower cost.
Reena M is an FSI Solutions Architect at AWS, specializing in analytics and generative AI-based workloads, helping capital markets and banking customers create secure, scalable, and efficient solutions on AWS. She architects cutting-edge data platforms and AI-powered applications that transform how financial institutions leverage cloud technologies. Beyond her technical pursuits, Reena is also a writer and enjoys spending time with her family.

Zhipu AI has open sourced the GLM-4.6V series as a pair of vision language models that treat images, video and tools as first class inputs for agents, not as afterthoughts bolted on top of text.

Model lineup and context length

The series has 2 models. GLM-4.6V is a 106B parameter foundation model for cloud and high performance cluster workloads. GLM-4.6V-Flash is a 9B parameter variant tuned for local deployment and low latency use.

GLM-4.6V extends the training context window to 128K tokens. In practice this supports roughly 150 pages of dense documents, 200 slide pages or one hour of video in a single pass because pages are encoded as images and consumed by the visual encoder.

Native multimodal tool use

The main technical change is native multimodal Function Calling. Traditional tool use in LLM systems routes everything through text. Images or pages are first turned into descriptions, the model calls tools using text arguments and then reads textual responses. This wastes information and increases latency.

GLM-4.6V introduces native multimodal Function Calling. Images, screenshots and document pages pass directly as tool parameters. Tools can return search result grids, charts, rendered web pages or product images. The model consumes those visual outputs and fuses them with text in the same reasoning chain. This closes the loop from perception to understanding to execution and is explicitly positioned as the bridge between visual perception and executable action for multimodal agents.

To support this, Zhipu AI extends the Model Context Protocol with URL based multimodal handling. Tools receive and return URLs that identify specific images or frames, which avoids file size limits and allows precise selection inside multi image contexts.

Rich text content, web search and frontend replication

Zhipu AI research team describes 4 canonical scenarios:

First, rich text content understanding and creation. GLM-4.6V reads mixed inputs such as papers, reports or slide decks and produces structured image text interleaved outputs. It understands text, charts, figures, tables and formulas in the same document. During generation it can crop relevant visuals or retrieve external images through tools, then run a visual audit step that filters low quality images and composes the final article with inline figures.

Second, visual web search. The model can detect user intent, plan which search tools to call and combine text to image and image to text search. It then aligns retrieved images and text, selects the relevant evidence and outputs a structured answer, for example a visual comparison of products or places.

Third, frontend replication and visual interaction. GLM-4.6V is tuned for design to code workflows. From a UI screenshot, it reconstructs pixel accurate HTML, CSS and JavaScript. Developers can then mark a region on the screenshot and issue natural language instructions, for example move this button left or change this card background. The model maps those instructions back to the code and returns an updated snippet.

Fourth, multimodal document understanding at long context. GLM-4.6V can read multi document inputs up to the 128K token context limit by treating pages as images. The research team reports a case where the model processes financial reports from 4 public companies, extracts core metrics and builds a comparison table, and a case where it summarises a full football match while keeping the ability to answer questions about specific goals and timestamps.

Architecture, data and reinforcement learning

The GLM-4.6V models belong to the GLM-V family and based on the tech report for GLM-4.5V and GLM-4.1V-Thinking. The research team highlights three main technical ingredients.

First, long sequence modeling. GLM-4.6V extends the training context window to 128K tokens and runs continual pre training on massive long context image text corpora. It uses compression alignment ideas from Glyph so that visual tokens can carry dense information that is aligned with language tokens.

Second, world knowledge enhancement. Zhipu AI team adds a billion scale multimodal perception and world knowledge dataset at pre training time. This covers layered encyclopedic concepts and everyday visual entities. The stated goal is to improve both basic perception and cross modal question answering completeness, not only benchmarks.

Third, agentic data synthesis and extended MCP. The research team generates large synthetic traces where the model calls tools, processes visual outputs and iterates on plans. They extend MCP with URL based multimodal handling and an interleaved output mechanism. The generation stack follows a Draft, Image Selection, Final Polish sequence. The model can autonomously call cropping or search tools between these stages to place images at the right positions in the output.

Tool invocation is part of the reinforcement learning objective. GLM-4.6V uses RL to align planning, instruction following and format adherence in complex tool chains.

Performance

https://z.ai/blog/glm-4.6v

Key Takeaways

GLM-4.6V is a 106B multimodal foundation model with a 128K token training context, and GLM-4.6V-Flash is a 9B variant optimized for local and low latency use.

Both models support native multimodal Function Calling so tools can consume and return images, video frames and document pages directly, which links visual perception to executable actions for agents.

GLM-4.6V is trained for long context multimodal understanding and interleaved generation, so it can read large mixed document sets and emit structured text with inline figures and tool selected images in one pass.

The series achieves state of the art performance on major multimodal benchmarks at similar parameter scales and is released as open source weights under the MIT license on Hugging Face and ModelScope.

Check out the Model Card on HF and Technical details. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The post Zhipu AI Releases GLM-4.6V: A 128K Context Vision Language Model with Native Tool Calling appeared first on MarkTechPost.

Jina AI has released Jina-VLM, a 2.4B parameter vision language model that targets multilingual visual question answering and document understanding on constrained hardware. The model couples a SigLIP2 vision encoder with a Qwen3 language backbone and uses an attention pooling connector to reduce visual tokens while preserving spatial structure. Among open 2B scale VLMs, it reaches state of the art results on multilingual benchmarks such as MMMB and Multilingual MMBench.

https://arxiv.org/pdf/2512.04032

Architecture, overlapping tiles with attention pooling connector

Jina-VLM keeps the standard VLM layout, but optimizes the vision side for arbitrary resolution and low token count. The vision encoder is SigLIP2 So400M/14 384, a 27 layer Vision Transformer with about 400M parameters. It processes 378×378 pixel crops into a 27×27 grid of 14×14 patches, so each tile produces 729 patch tokens.

To handle high resolution images, the model does not resize the full input to a single square. Instead, it constructs a grid of up to 12 overlapping tiles along with a global thumbnail. Each tile is a 378×378 crop, adjacent tiles overlap by 112 pixels, and the stride between tile origins is 266 pixels. A 4×3 grid covers an effective resolution of 1176×910 pixels before downscaling larger images to fit inside the tile budget.

The core design is the vision language connector. Rather than using the final ViT layer, Jina-VLM concatenates features from two intermediate layers, the third from last and ninth from last, that correspond to layers 24 and 18. This combines high level semantics and mid level spatial detail. The connector then applies attention pooling over 2×2 patch neighborhoods. It computes a mean pooled query for each 2×2 region, attends over the full concatenated feature map, and outputs a single pooled token per neighborhood. This reduces 729 visual tokens per tile to 182 tokens, which is a 4 times compression. A SwiGLU projection maps the pooled features to the Qwen3 embedding dimension.

With the default 12 tile configuration plus thumbnail, a naive connector would feed 9,477 visual tokens into the language model. Attention pooling cuts this to 2,366 visual tokens. The ViT compute does not change, but for the language backbone this yields about 3.9 times fewer prefill FLOPs and 4 times smaller KV cache. When including the shared ViT cost, the overall FLOPs drop by about 2.3 times for the default setting.

The language decoder is Qwen3-1.7B-Base. The model introduces special tokens for images, with <im_start> and <im_end> around the tile sequence and <im_col> to mark rows in the patch grid. Visual tokens from the connector and text embeddings are concatenated and passed to Qwen3 to generate answers.

Training pipeline and multilingual data mix

Training proceeds in 2 stages. All components, encoder, connector and decoder, are updated jointly, without freezing. The full corpus contains about 5M multimodal samples and 12B text tokens across more than 30 languages. Roughly half of the text is English, and the rest covers high and mid resource languages such as Chinese, Arabic, German, Spanish, French, Italian, Japanese and Korean.

Stage 1 is alignment training. The goal is cross language visual grounding, not instruction following. The team uses caption heavy datasets PixmoCap and PangeaIns, which span natural images, documents, diagrams and infographics. They add 15 percent text only data from the PleiAS common corpus to control degradation on pure language tasks. The connector uses a higher learning rate and shorter warmup than the encoder and decoder to speed up adaptation without destabilizing the backbones.

Stage 2 is instruction fine tuning. Here Jina VLM learns to follow prompts for visual question answering and reasoning. The mix combines LLaVA OneVision, Cauldron, Cambrian, PangeaIns and FineVision, plus Aya style multilingual text only instructions. The Jina research team first train for 30,000 steps with single source batches, then for another 30,000 steps with mixed source batches. This schedule stabilizes learning in the presence of very heterogeneous supervision.

Across pretraining and fine tuning, the model sees about 10B tokens in the first stage and 37B tokens in the second stage, with a total of roughly 1,300 GPU hours reported for the main experiments.

Benchmark profile, 2.4B model with multilingual strength

On standard English VQA tasks that include diagrams, charts, documents, OCR and mixed scenes, Jina-VLM reaches an average score of 72.3 across 8 benchmarks. These are AI2D, ChartQA, TextVQA, DocVQA, InfoVQA, OCRBench, SEED Bench 2 Plus and CharXiv. This is the best average among the 2B scale comparison models in this research paper from Jina AI.

On multimodal comprehension and real world understanding tasks, the model scores 67.4 on the multimodal group, which includes MME, MMB v1.1 and MMStar. It scores 61.9 on the real world group, which includes RealWorldQA, MME RealWorld and R Bench, and it reaches 68.2 accuracy on RealWorldQA itself, which is the best result among the baselines considered.

https://arxiv.org/pdf/2512.04032

Multi image reasoning is a weaker area. On BLINK, MuirBench and MMT, Jina-VLM reaches an average of 47.3. The research team point to limited multi-image training data as the reason. In contrast, hallucination control is strong. On the POPE benchmark, which measures object hallucination, the model scores 90.3, the best score in the comparison table.

For mathematical and structured reasoning, the model uses the same architecture, without thinking mode. It reaches 59.5 on MMMU and an overall math score of 33.3 across MathVista, MathVision, MathVerse, WeMath and LogicVista. Jina-VLM is comparable to InternVL3-2B on this set and clearly ahead of Qwen2-VL-2B, while InternVL3.5-2B remains stronger due to its larger scale and more specialized math training.

On pure text benchmarks, the picture is mixed. The research team reports that Jina-VLM keeps most of the Qwen3-1.7B performance on MMLU, GSM 8K, ARC C and HellaSwag. However, MMLU-Pro drops from 46.4 for the base model to 30.3 after multimodal tuning. The research team attribute this to instruction tuning that pushes the model toward very short answers, which clashes with the long multi step reasoning required by MMLU Pro.

The main highlight is multilingual multimodal understanding. On MMMB across Arabic, Chinese, English, Portuguese, Russian and Turkish, Jina-VLM reaches an average of 78.8. On Multilingual MMBench across the same languages, it reaches 74.3. The research team reports these as state of the art averages among open 2B scale VLMs.

Comparison Table

ModelParamsVQA AvgMMMBMulti. MMBDocVQAOCRBenchJina-VLM2.4B72.378.874.390.6778Qwen2-VL-2B2.1B66.471.369.489.2809Qwen3-VL-2B2.8B71.675.072.392.3858InternVL3-2B2.2B69.273.671.987.4835InternVL3.5-2B2.2B71.674.670.988.5836

Key Takeaways

Jina-VLM is a 2.4B parameter VLM that couples SigLIP2 So400M as vision encoder with Qwen3-1.7B as language backbone through an attention pooling connector that cuts visual tokens by 4 times while keeping spatial structure.

The model uses overlapping 378×378 tiles, 12 tiles plus a global thumbnail, to handle arbitrary resolution images up to roughly 4K, then feeds only pooled visual tokens to the LLM which reduces prefill FLOPs and KV cache size by about 4 times compared to naive patch token usage.

Training uses about 5M multimodal samples and 12B text tokens across nearly 30 languages in a 2 stage pipeline, first alignment with caption style data, then instruction fine tuning with LLaVA OneVision, Cauldron, Cambrian, PangeaIns, FineVision and multilingual instruction sets.

On English VQA, Jina-VLM reaches 72.3 average across 8 VQA benchmarks, and on multilingual multimodal benchmarks it leads the open 2B scale class with 78.8 on MMMB and 74.3 on Multilingual MMBench while keeping competitive text only performance.

Check out the Paper, Model on HF and Technical details. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The post Jina AI Releases Jina-VLM: A 2.4B Multilingual Vision Language Model Focused on Token Efficient Visual QA appeared first on MarkTechPost.

What comes after Transformers? Google Research is proposing a new way to give sequence models usable long term memory with Titans and MIRAS, while keeping training parallel and inference close to linear.

Titans is a concrete architecture that adds a deep neural memory to a Transformer style backbone. MIRAS is a general framework that views most modern sequence models as instances of online optimization over an associative memory.

Why Titans and MIRAS?

Standard Transformers use attention over a key value cache. This gives strong in context learning, but cost grows quadratically with context length, so practical context is limited even with FlashAttention and other kernel tricks.

Efficient linear recurrent neural networks and state space models such as Mamba-2 compress the history into a fixed size state, so cost is linear in sequence length. However, this compression loses information in very long sequences, which hurts tasks such as genomic modeling and extreme long context retrieval.

Titans and MIRAS combine these ideas. Attention acts as a precise short term memory on the current window. A separate neural module provides long term memory, learns at test time, and is trained so that its dynamics are parallelizable on accelerators.

https://research.google/blog/titans-miras-helping-ai-have-long-term-memory/

Titans, a neural long term memory that learns at test time

The Titans research paper introduces a neural long term memory module that is itself a deep multi layer perceptron rather than a vector or matrix state. Attention is interpreted as short term memory, since it only sees a limited window, while the neural memory acts as persistent long term memory.

For each token, Titans defines an associative memory loss

ℓ(Mₜ₋₁; kₜ, vₜ) = ‖Mₜ₋₁(kₜ) − vₜ‖²

where Mₜ₋₁ is the current memory, kₜ is the key and vₜ is the value. The gradient of this loss with respect to the memory parameters is the “surprise metric”. Large gradients correspond to surprising tokens that should be stored, small gradients correspond to expected tokens that can be mostly ignored.

The memory parameters are updated at test time by gradient descent with momentum and weight decay, which together act as a retention gate and forgetting mechanism.To keep this online optimization efficient, the research paper shows how to compute these updates with batched matrix multiplications over sequence chunks, which preserves parallel training across long sequences.

Architecturally, Titans uses three memory branches in the backbone, often instanced in the Titans MAC variant:

a core branch that performs standard in context learning with attention

a contextual memory branch that learns from the recent sequence

a persistent memory branch with fixed weights that encodes pretraining knowledge

The long term memory compresses past tokens into a summary, which is then passed as extra context into attention. Attention can choose when to read that summary.

Experimental results for Titans

On language modeling and commonsense reasoning benchmarks such as C4, WikiText and HellaSwag, Titans architectures outperform state of the art linear recurrent baselines Mamba-2 and Gated DeltaNet and Transformer++ models of comparable size. The Google research attribute this to the higher expressive power of deep memory and its ability to maintain performance as context length grows. Deep neural memories with the same parameter budget but higher depth give consistently lower perplexity.

For extreme long context recall, the research team uses the BABILong benchmark, where facts are distributed across very long documents. Titans outperforms all baselines, including very large models such as GPT-4, while using many fewer parameters, and scales to context windows beyond 2,000,000 tokens.

The research team reports that Titans keeps efficient parallel training and fast linear inference. Neural memory alone is slightly slower than the fastest linear recurrent models, but hybrid Titans layers with Sliding Window Attention remain competitive on throughput while improving accuracy.

https://arxiv.org/pdf/2504.13173

MIRAS, a unified framework for sequence models as associative memory

The MIRAS research paper, “It’s All Connected: A Journey Through Test Time Memorization, Attentional Bias, Retention, and Online Optimization,” generalizes this view. It observes that modern sequence models can be seen as associative memories that map keys to values while balancing learning and forgetting.

MIRAS defines any sequence model through four design choices:

Memory structure for example a vector, linear map, or MLP

Attentional bias the internal loss that defines what similarities the memory cares about

Retention gate the regularizer that keeps the memory close to its past state

Memory algorithm the online optimization rule, often gradient descent with momentum

Using this lens, MIRAS recovers several families:

Hebbian style linear recurrent models and RetNet as dot product based associative memories

Delta rule models such as DeltaNet and Gated DeltaNet as MSE based memories with value replacement and specific retention gates

Titans LMM as a nonlinear MSE based memory with local and global retention optimized by gradient descent with momentum

Crucially, MIRAS then moves beyond the usual MSE or dot product objectives. The research team constructs new attentional biases based on Lₚ norms, robust Huber loss and robust optimization, and new retention gates based on divergences over probability simplices, elastic net regularization and Bregman divergence.

From this design space, the research team instantiate three attention free models:

Moneta uses a 2 layer MLP memory with Lₚ attentional bias and a hybrid retention gate based on generalized norms

Yaad uses the same MLP memory with Huber loss attentional bias and a forget gate related to Titans

Memora uses regression loss as attentional bias and a KL divergence based retention gate over a probability simplex style memory.

These MIRAS variants replace attention blocks in a Llama style backbone, use depthwise separable convolutions in the Miras layer, and can be combined with Sliding Window Attention in hybrid models. Training remains parallel by chunking sequences and computing gradients with respect to the memory state from the previous chunk.

In research experiments, Moneta, Yaad and Memora match or surpass strong linear recurrent models and Transformer++ on language modeling, commonsense reasoning and recall intensive tasks, while maintaining linear time inference.

Key Takeaways

Titans introduces a deep neural long term memory that learns at test time, using gradient descent on an L2 associative memory loss so the model selectively stores only surprising tokens while keeping updates parallelizable on accelerators.

Titans combines attention with neural memory for long context, using branches like core, contextual memory and persistent memory so attention handles short range precision and the neural module maintains information over sequences beyond 2,000,000 tokens.

Titans outperforms strong linear RNNs and Transformer++ baselines, including Mamba-2 and Gated DeltaNet, on language modeling and commonsense reasoning benchmarks at comparable parameter scales, while staying competitive on throughput.

On extreme long context recall benchmarks such as BABILong, Titans achieves higher accuracy than all baselines, including larger attention models such as GPT 4, while using fewer parameters and still enabling efficient training and inference.

MIRAS provides a unifying framework for sequence models as associative memories, defining them by memory structure, attentional bias, retention gate and optimization rule, and yields new attention free architectures such as Moneta, Yaad and Memora that match or surpass linear RNNs and Transformer++ on long context and reasoning tasks.

Check out the Technical details. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The post From Transformers to Associative Memory, How Titans and MIRAS Rethink Long Context Modeling appeared first on MarkTechPost.

When critical services depend on quick action, from the safety of vulnerable children to environmental protection, you need working AI solutions in weeks, not years. Amazon recently announced an investment of up to $50 billion in expanded AI and supercomputing infrastructure for US government agencies, demonstrating both the urgency and commitment from Amazon Web Services (AWS) to accelerating public sector innovation. The AWS Generative AI Innovation Center is already making this happen, consistently delivering production-ready solutions for government organizations.
What makes this time different
The convergence of three factors makes this technology moment different:

Mission urgency – Public sector organizations currently face the challenge of managing both growing workloads in mission-critical areas, such as veterans’ benefits claims and bridge safety inspections, and workforce and budget limitations.
Technology readiness – Production-ready AI solutions can now be deployed securely and at scale, with unprecedented compute capacity being built specifically for US government requirements.
Proven success models – Early adopters have demonstrated that rapid AI implementation is possible in government settings, creating blueprints for others to follow.

Drawing from over a thousand implementations, the Generative AI Innovation Center combines AWS infrastructure and security conformance to help you transform mission delivery.

Accelerating real-world innovation
Public sector organizations working to improve mission speed and effectiveness can collaborate with the Innovation Center to develop targeted solutions. These three case studies show this approach in action.
AI systems that support critical care to protect vulnerable children
When protecting a child’s welfare, having key information surface at exactly the right moment is crucial. Systems must work reliably, every time.
This was the challenge the Miracle Foundation faced when managing foster care caseloads globally. In the span of weeks, the Innovation Center worked alongside caseworkers to build a production AI assistant that analyzes case files, flags urgent situations, and recommends evidence-based interventions tailored to each child’s unique circumstances.
“When a caseworker misses an urgent signal in a child’s file, it can have life-changing consequences,” explains Innovation Center strategist Brittany Roush. “We were building a system that needed to surface critical information at exactly the right moment.”
The solution aims to help caseworkers make faster, more informed decisions for vulnerable children around the world. It also includes built-in enterprise-grade security, designed for scalability and delivered with comprehensive knowledge transfer so the Miracle Foundation team can fully manage and evolve their system.
It’s important to start with actual users on day one. The Miracle Foundation team interfaced directly with caseworkers to understand workflows before writing a single line of code. This user-first approach removed months of work to gather requirements and iterate through revisions.
Innovation at institutional scale
The University of Texas at Austin (UT Austin) approached the Innovation Center about personalized academic support for 52,000 students. The team delivered UT Sage, a production AI tutoring service designed by learning scientists and trained by faculty, which is now in open beta across the UT Austin campus. Unlike generic AI tools, UT Sage provides custom, course-specific support while maintaining academic integrity standards. “It’s like having a knowledgeable teaching assistant available whenever you need help,” one student reported during testing.
“The UT Sage project empowers our faculty to create personalized learning tools, designed to motivate student engagement,” said Julie Schell, Assistant Vice Provost and Director of the Office of Academic Technology. “With the potential to deploy across hundreds of courses, we are aiming to enhance learning outcomes and reduce the time and effort required to design student-centered, high-quality course materials.”
Build flexible foundations, not point solutions. The team architected UT Sage as a service that faculty could adapt to specific courses. This extensible design enabled institutional scale from day one, avoiding the trap of a successful pilot that never scales, which can plague technology projects.
Transforming government speed with the EPA
The U.S. Environmental Protection Agency partnered with the innovation center to transform document processing workflows that used to take weeks or months. The team, in partnership with the EPA, delivered two breakthrough solutions that demonstrate both the team’s velocity and increasing technical complexity:

Chemical risk assessment acceleration – An intelligent document processing system that evaluates research studies against predetermined scientific criteria. What once required hours of manual review by EPA scientists now takes minutes. The system achieved an 85% reduction in processing time while maintaining 85% accuracy. Processing 250 documents costs the team $40 through the system, compared to requiring 500 hours of scientist time manually.
Federal Insecticide, Fungicide, and Rodenticide Act (FIFRA) application reviews – Automated creation of data evaluation records (DERs) from health and safety studies for pesticide applications under FIFRA. This process traditionally took EPA reviewers 4 months of manual work. The AI solution now generates these critical regulatory documents in seconds, achieving a 99% cost reduction while potentially accelerating approval timelines for safe pesticide products.

Both solutions incorporate rigorous human-in-the-loop review processes to maintain scientific integrity and regulatory compliance alignment. EPA scientists oversee AI-generated assessments, but they can now focus their expertise on analysis and decision-making rather than manual data processing.
“We’re not replacing scientific judgment,” explained an EPA team member. “We’re eliminating the tedious work so our scientists can spend more time on what matters most—protecting public health and the environment.”
The EPA cases demonstrate that AI augmentation can deliver both speed and trust. The team designed review workflows into the architecture to improve trust, making the systems immediately acceptable to scientific staff and leadership.
Strategies to increase the pace of innovation
Experts at the Innovation Center have developed several strategies to help organizations excel with generative AI. To facilitate building your own production systems and increase the pace of innovation, follow these best practices:

Build on day 1, not week 6 – Traditional projects spend months on requirements and architecture. The Innovation Center starts building immediately, using extensive libraries of reusable, secure infrastructure-as-code (IaC) components. They also use tools such as Kiro, an AI integrated development environment (IDE) that efficiently converts developer prompts into detailed specifications and working code. This approach has been refined with each engagement, meaning the team is building increasingly complex use cases faster than ever before. Access to the expanded government AI infrastructure of AWS can further accelerate this development process, so you can tackle increasingly sophisticated use cases.
Get the right people on your team – Each engagement brings together scientists, architects, security experts, and domain specialists who understand public sector missions. This cross-functional composition minimizes the typical back-and-forth that often complicates requirement gathering and refinement. Everyone who’s needed to make decisions is already in the discussion, collaboratively working toward a common goal.
Knowledge transfer happens throughout, not at the end – Don’t wait to think about technology hand-offs. Advancing a project to the next team without prior coordination is rarely an effective strategy. The deep collaboration between stakeholders working alongside Innovation Center experts happens throughout development. Knowledge transfer occurs naturally in daily collaboration, with formal documentation being handed off at the end. The Innovation Center team then continues to support in an advisory capacity until the solution goes into production.
Harness the secure and reliable infrastructure and services of AWS – For public sector organizations, moving fast can’t mean compromising on security or compliance. Every solution is architected on secure AWS infrastructure with the ability to meet even stringent Federal Risk and Authorization Management Program (FedRAMP) High requirements. The Innovation Center follows a secure-by-design approach where compliance alignment is woven into the entire development lifecycle. By making compliance alignment concurrent, not sequential, the team demonstrates that security and speed aren’t trade-offs. The upcoming expansion of the AWS government cloud infrastructure further strengthens these security and compliance capabilities, providing you with one of the most comprehensive and secure AI computing environments.

Next steps in public sector AI
Every case study in this post started with a specific, pressing challenge. Each example achieved institutional scale by delivering value quickly, not by waiting for the perfect moment. Start with one persistent operational need, deliver results in weeks, then expand. With the AWS investment of up to $50 billion in purpose-built government AI infrastructure, these transformations can now happen at even greater scale and speed. Each successful engagement creates a blueprint for the next, continuously expanding what’s possible for public sector AI.
Learn more about the AWS Generative AI Innovation Center and how they’re helping public sector organizations turn AI potential into production reality.

About the authors
Kate Zimmerman serves as the Generative AI Innovation Center Geo Leader for Worldwide Public Sector at AWS. Kate leads a team of generative AI strategists and scientists, architecting innovative solutions for public sector organizations globally. Her role combines strategic leadership with hands-on technical expertise, and she works directly with Director, VP, and C-level executives to drive GenAI adoption and deliver mission-critical outcomes. With 13+ years of experience spanning commercial cloud, defense, national security, and aerospace, Kate brings a unique perspective to driving transformative AI/ML solutions. Previously, as Chief Scientist & VP of Data and Analytics at HawkEye 360, she led 50+ developers, engineers, and scientists to launch the company’s first production AI/ML capabilities. Her tenure at AWS included leadership roles as Senior Manager & Principal Architect of the ML Solutions Lab, where she accelerated AI/ML adoption among national security customers, and Senior Solutions Architect supporting the National Reconnaissance Office. Kate also served in the USAF on active duty for 5 years developing advance satellite systems and continues to serve as a reservist supporting strategic AI/ML initiatives with the USAF 804th Test Group.
Sri Elaprolu serves as Director of the AWS Generative AI Innovation Center, where he leverages nearly three decades of technology leadership experience to drive artificial intelligence and machine learning innovation. In this role, he leads a global team of machine learning scientists and engineers who develop and deploy advanced generative and agentic AI solutions for enterprise and government organizations facing complex business challenges. Throughout his nearly 13-year tenure at AWS, Sri has held progressively senior positions, including leadership of ML science teams that partnered with high-profile organizations such as the NFL, Cerner, and NASA. These collaborations enabled AWS customers to harness AI and ML technologies for transformative business and operational outcomes. Prior to joining AWS, he spent 14 years at Northrop Grumman, where he successfully managed product development and software engineering teams. Sri holds a Master’s degree in Engineering Science and an MBA with a concentration in general management, providing him with both the technical depth and business acumen essential for his current leadership role.

Today, we are pleased to announce a new integration between Amazon Quick Research and S&P Global. This integration brings both S&P Global Energy news, research, and insights and S&P Global Market Intelligence data to Quick Research customers in one deep research agent.
The S&P Global integration extends the capabilities of Quick Research so that business professionals can analyze multiple data sources—including global energy news and premium financial intelligence—in one workspace, eliminating the need to switch between platforms and transforming weeks of research into minutes of focused insight generation. Quick Suite connects information across internal repositories, popular applications, AWS services, and through Model Context Protocol (MCP) integrations, to over 1,000 apps. This agentic AI application is reshaping how work gets done by transforming how teams find insights, conduct deep research, automate tasks, visualize data, and take actions across apps.
In this post, we explore S&P Global’s data sets and the solution architecture of the integration with Quick Research.
Solution overview
S&P Global has pioneered two MCP server implementations on AWS so organizations can easily integrate trusted financial services and energy content into AI-powered workflows while maintaining the quality, security, and reliability that business leaders demand.

“Our collaboration with AWS expands how S&P Global delivers trusted intelligence through the next generation of agentic AI experiences. By working alongside leading AI companies, our goal is to ensure customers can access our trusted data and insights wherever their workflows take place.” 
– Bhavesh Dayalji, Chief AI Officer of S&P Global and CEO of Kensho.

S&P Global Energy: Comprehensive commodity and energy intelligence
The S&P Global Energy integration, now available in Amazon Quick Research, utilizes an AI Ready Data MCP server to deliver comprehensive access to commodity and energy market intelligence spanning Oil, Gas, Power, Metals, Clean Energy, Agriculture, and Shipping sectors across global markets. Built on S&P Global’s reputation as a trusted market authority, the MCP server uses hundreds of thousands of expert-created documents including analyses, commentaries, and news articles reflecting decades of industry expertise.
The solution provides a unique multi-horizon perspective, offering intelligence from daily market updates to one-year outlooks and extending to 20+ year scenario analyses. With data refreshing every 30 minutes, business leaders gain near real-time access to commodity and energy intelligence, dramatically accelerating decision velocity when exploring regulatory challenges, investment opportunities, or environmental implications.
S&P Global Market Intelligence: Trusted financial intelligence
The S&P Global Market Intelligence integration, now available in Amazon Quick Research, uses the Kensho LLM-ready API MCP server developed by Kensho, S&P Global’s AI innovation hub. This MCP server makes trusted financial data accessible through natural language queries, integrating seamlessly with Amazon Quick Research. Financial professionals can access S&P Capital IQ Financials, earnings call transcripts, company information, transactions and more, simply by asking questions.
The Kensho solution addresses a critical challenge in financial services: making vast repositories of financial data immediately accessible without requiring complex query languages or technical expertise. Engineering, product, and business teams can save significant time and resources by transforming what once required hours of data extraction into conversational queries that return precise, trusted information in seconds.
Solution architecture
S&P Global’s MCP server architecture is shown in the following diagram. When using one of the S&P integrations, traffic flows from Quick Research through an Amazon API Gateway to an AWS Application Load Balancer with the MCP services hosted on Amazon Elastic Kubernetes Service (Amazon EKS). The MCP server uses data hosted in Amazon S3 and AWS Relational Database Service for PostgreSQL for structured data, and Amazon OpenSearch Service for vector storage. This architecture delivers enterprise-ready MCP servers with defense-in-depth security, automated scaling, and comprehensive observability.

MCP is an open standard that supports seamless communication between AI agents and external data sources, tools, and services. MCP operates on a client-server architecture where MCP servers handle tool calls, typically consisting of multiple API calls and expose business logic implementations as callable functions. This enables AI agents to discover capabilities dynamically, negotiate features, and share context securely with all critical requirements for enterprise-grade applications.
S&P Global’s solution has the following key building blocks:

Automated data pipeline with Amazon Bedrock: At the heart of the solution is a Retrieval Augmented Generation (RAG) data ingestion pipeline using Amazon Bedrock. This pipeline transforms raw market data into AI Ready Data. Documents from S&P Global’s proprietary repositories undergo preprocessing, chunking, and enrichment before being converted into vector embeddings using Bedrock hosted Cohere Embed model. The ingestion pipeline runs on a scheduled basis, refreshing the OpenSearch vector store every 30 minutes for near real-time access to the energy data.
Vector and semantic search: Amazon OpenSearch serves as the vector database, storing embeddings generated by Bedrock and enabling semantic search capabilities across S&P Global’s energy data. The OpenSearch vector store is optimized for high-dimensional vector operations, supporting rapid similarity searches that power the MCP servers’ ability to retrieve contextually relevant information in response to natural language queries.
Resilience and scale: This solution uses Amazon EKS to host all MCP server solutions with two production clusters enabling traffic splitting and failover capabilities. This dual-cluster approach provides continuous availability even during unexpected failures. Both the Cluster Autoscaler and Horizontal Pod Autoscaler enable dynamic scaling based on demand. The MCP servers are built with the FastMCP framework, providing high-performance HTTP endpoints that comply with the Streamable HTTP Transport specification required by the MCP protocol.
Security: Security is built-in to every layer of the solution. API Gateway serves as the endpoint for MCP server access. S&P Global’s enterprise identity provider is used for OAuth authentication. API Gateway is further secured with AWS Web Application Firewall (WAF) with advanced threat detection. AWS IAM roles and policies enforce least privilege principles, so that each component has only the permissions it requires. AWS Secrets Manager securely stores credentials for accessing resources and AWS services. AWS Security Groups and VPC configurations provide network isolation, while TLS 1.2+ with AWS Certificate Manager validates all data in transit remains encrypted. This multi-layered security includes defense-in-depth security controls.
Observability: Amazon CloudWatch provides centralized logging, metrics collection, and real-time monitoring of the entire pipeline from data ingestion through MCP server responses. AWS CloudTrail captures detailed API activity logs and audit trails, essential for compliance in regulated industries.

Conclusion
Together, these MCP servers built on AWS and integrated into Amazon Quick Research demonstrates S&P Global’s vision for the future of financial services and energy intelligence: maintaining the trust, accuracy, and depth that business leaders require while embracing the transformative potential of AI to make that intelligence more accessible, actionable, and integrated into modern workflows.
Ready to get started? Please refer to Quick Research Third Party Data for more details.

About the authors
Jon Einkauf is a Product leader at AWS based in Seattle, where he focuses on building AI-powered tools that help businesses synthesize information and accelerate research. With over a decade of experience at Amazon spanning digital health, cloud computing, and AI products, he has led cross-functional teams in product management, engineering, and design to deliver innovative solutions for customers worldwide.
Prasanth Ponnoth is an AWS solutions architect supporting global financial services with more than 20 years of industry and technology experience with cloud migration, modernization and building distributed systems at scale. His areas of interests are machine learning, containers/ Kubernetes and open-source technologies. In AWS, he is part of the machine learning technical field community and focusing on Amazon Bedrock, Amazon SageMaker AI, Amazon Bedrock AgentCore services.
Brandon Pominville is a Senior Solutions Architect at AWS based in New York, where he works with global financial services customers to build secure, scalable data and AI platforms in the cloud. With over 20 years of experience across financial services, enterprise data platforms, and cloud computing, he specializes in translating business requirements into technical solutions. Outside of work, Brandon enjoys spending time with his family outdoors or on a cruise ship, and playing volleyball.

AgentCore Gateway now supports API GatewayAs organizations explore the possibilities of agentic applications, they continue to navigate challenges of using enterprise data as context in invocation requests to large language models (LLMs) in a manner that is secure and aligned with enterprise policies. To help standardize and secure those interactions, many organizations are using the Model Context Protocol (MCP) specification, which defines how agentic applications can securely connect to data sources and tools.
While MCP has been advantageous for net new use cases, organizations also navigate challenges with bringing their existing API estate into the agentic era. MCP can certainly wrap existing APIs, but it requires additional work, translating requests from MCP to RESTful APIs, making sure security is maintained through the entire request flow, and applying the standard observability required for production deployments.
Amazon Bedrock AgentCore Gateway now supports Amazon API Gateway as a target, translating MCP requests to AgentCore Gateway into RESTful requests to API Gateway. You can now expose both new and existing API endpoints to agentic applications using MCP, with built-in security and observability. This post covers these new capabilities and shows how to implement them.
What’s new: API Gateway support in AgentCore Gateway
AgentCore Gateway now supports API Gateway targets in addition to existing target types (Lambda functions, OpenAPI schemas, Smithy models, and MCP servers).

Our customers have successfully built extensive API ecosystems using API Gateway, connecting backends across numerous applications. As enterprises advance toward next-generation agentic applications, the natural evolution is to expose these existing APIs and backend tools to AI-powered systems, enabling seamless integration between established infrastructure and modern intelligent agents.
This integration between AgentCore Gateway and API Gateway simplifies the connection between API Gateway and AgentCore Gateway. It allows you to directly target API Gateway, so that you don’t need to export API Gateway APIs as an OpenAPI 3 specification and then add it to AgentCore Gateway as an OpenAPI target.
With this integration, a new API_GATEWAY target type will be added to AgentCore Gateway, eliminating the manual export/import process. REST API owners can add their API as an AgentCore Gateway target with a few console interactions or a single CLI command to expose their existing REST API as MCP tools using AgentCore Gateway. API consumers can then connect AI agents with these REST APIs through the Model Context Protocol (MCP) and power their workflows with AI integration. Your agentic applications can now connect to your new or existing API Gateway API. This integration between AgentCore Gateway and API Gateway supports IAM and API key authorization.

Both AgentCore Gateway and API Gateway have integrations with Amazon CloudWatch Logs, AWS CloudTrail, and AWS X-Ray for observability. Agent developers using this new capability between AgentCore Gateway and API Gateway can use these observability tools.
Walkthrough
This post shows you how to set up an existing REST API with API Gateway as a target for AgentCore Gateway. With this integration you can use your existing REST APIs as a tool for your agentic applications exposed using AgentCore Gateway.
Prerequisites
For this example, you need the following:

An AWS account with an existing REST API in API Gateway.
An Identity and Access Management (IAM) role or user with enough permissions to create an AgentCore Gateway and set up an API Gateway target.

You can create gateways and add targets in multiple ways:

AWS Management Console
AWS SDK for Python (Boto3)
AWS Command Line Interface (AWS CLI)
AgentCore starter toolkit for fast and straightforward set up

This post uses Boto3 for setting up the integration between AgentCore Gateway and API Gateway. For an interactive walkthrough, you can use the Jupyter Notebook sample on GitHub.
Set up prerequisites for inbound and outbound authorization.
Inbound authorization authenticates incoming user requests. Outbound authorization helps AgentCore Gateway to securely connect to gateway targets, such as an API Gateway, on behalf of the authenticated user.

For API Gateway as a target, AgentCore Gateway supports the following types of outbound authorization:

No authorization (not recommended) – Some target types provide you the option to bypass outbound authorization. We do not recommend this less secure option.
IAM-based outbound authorization – Use the gateway service role to authorize access to the gateway target with AWS Signature Version 4 (Sig V4).
API key – Use the API key, which is set up using AgentCore Identity to authorize access to API Gateway target. API keys created using an API Gateway mapped with API Gateway usage plans, helps you monitor and control API usage. Please refer to this documentation for more details.

Create an IAM role with the trust policy from the documentation.

For Outbound Authorization with IAM-based authorization, the policy should include execute-api:Invoke permission. Sample inline policy:

{
“Version”: “2012-10-17”,
“Statement”: [
{
“Action”: [
“execute-api:Invoke”,
],
“Resource”: ” “arn:aws:execute-api:{AWS_Region}:{AWS_Account_ID}:api-id/stage/METHOD_HTTP_VERB/resource-path”,
“Effect”: “Allow”
}
]
}

For API key authorization, you can create an API key (see the API Gateway documentation) and associate it with your API Gateway usage plan. Then create an API key credential provider with AgentCore Identity.

Once done, update the policy as described in the AgentCore Gateway documentation.
Create an AgentCore Gateway
When using the AgentCore starter toolkit, you can create a gateway with a default authorization configuration using Amazon Cognito for JWT-based inbound authorization.

import boto3
gateway_client = boto3.client(‘bedrock-agentcore-control’)
auth_config = {
“customJWTAuthorizer”: {
“allowedClients”: [‘<cognito_client_id>’], # Client MUST match with the ClientId configured in Cognito. Example: 7rfbikfsm51j2fpaggacgng84g
“discoveryUrl”: <cognito_oauth_discovery_url>
}
}
create_response = gateway_client.create_gateway(
name=’sample-ac-gateway’,
roleArn='<IAM_Role_ARN>’, # The IAM Role must have permissions to create/list/get/delete Gateway
protocolType=’MCP’,
protocolConfiguration={
‘mcp’: {
‘supportedVersions’: [‘2025-03-26’],
‘searchType’: ‘SEMANTIC’
}
},
authorizerType=’CUSTOM_JWT’,
authorizerConfiguration=auth_config,
description=’AgentCore Gateway with API Gateway target’
)
print(create_response)
# Retrieve the GatewayID used for GatewayTarget creation
gatewayID = create_response[“gatewayId”]
gatewayURL = create_response[“gatewayUrl”]
print(gatewayID)

This returns GATEWAY_ID that you will need to create the gateway target.
Create an AgentCore Gateway target

Create a target configuration
To create an API gateway target, you need to specify the following as the part of target configuration:

toolFilters: Use this to determine what resources on the REST API will be exposed as tool on the gateway. Filters also support wildcards in the filterPath.
toolOverrides (optional): Use this to allow users to override tool names and description. You must specify explicit paths and methods.
restApiId: Use this to pass API Gateway ID.

Below are a few examples of target configurations:
Example 1
This exposes GET & POST /pets, GET /pets/{petId} to the gateway and overrides their tool names and descriptions.

{
“mcp”: {
“apiGateway”: {
“restApiId”: “<api-id>”,
“stage”: “<stage>”,
“apiGatewayToolConfiguration”: {
“toolFilters”: [
{
“filterPath”: “/pets”,
“methods”: [“GET”,”POST”]
},
{
“filterPath”: “/pets/{petId}”,
“methods”: [“GET”]
}
],
“toolOverrides” : [
{
“name”: “ListPets”,
“path”: “/pets”,
“method”: “GET”,
“description”:”Retrieves all the available Pets.”
},
{
“name”: “AddPet”,
“path”: “/pets”,
“method”: “POST”,
“description”:”Add a new pet to the available Pets.”
},
{
“path”: “/pets/{petId}”,
“method”: “GET”,
“name”: “GetPetById”,
“description”: “Retrieve a specific pet by its ID”
}
]
}
}
}
}

Example 2
This will expose GET /pets but also GET /pets/{petId} or anything under /pets. Since toolOverrides is not specified, it will use the resource description from API Gateway.

{
“mcp”: {
“apiGateway”: {
“restApiId”: “<api-id>”,
“stage”: “<stage>”,
“apiGatewayToolConfiguration”: {
“toolFilters”: [
{
“filterPath”: “/pets/*”,
“methods”: [“GET”]
}
]
}
}
}
}

Credential provider configuration
When creating a target, you also need to specify the target’s outbound authorization using a credential provider configuration. As discussed above, there are three types of credential providers:
GATEWAY_IAM_ROLE
This uses the ROLE_ARN you specified when creating the gateway. Define the credential provider configuration as follows:

[
{
“credentialProviderType”: “GATEWAY_IAM_ROLE”
}
]

API_KEY
This requires the creation of an API key credential provider with AgentCore Identity.

[
{
“credentialProviderType”: “API_KEY”,
“credentialProvider”: {
“apiKeyCredentialProvider”: {
“providerArn”: “<provider-arn>”,
“credentialParameterName”: “x-api-key”, // optional
“credentialPrefix”: “abc”, // optional, prefix is added to the API key when sending it to the target endpoint
“credentialLocation”: “HEADER” //optional, specifies where in the request the API key should be placed
}
}
}
]

NO_AUTH
NO_AUTH can be configured by not specifying a credential provider configuration while creating the AgentCore Gateway target. This is not recommended.
Create an AgentCore Gateway target
Now configure your REST API as a gateway target:

import boto3
gateway_client = boto3.client(‘bedrock-agentcore-control’)
create_gateway_target_response = gateway_client.create_gateway_target(
name=’api-gateway-target’,
gatewayIdentifier='<gateway_ID>’,
targetConfiguration=[< your_target_configuration>],
credentialProviderConfigurations=[<your_credential_config>]
)
print(create_gateway_target_response)
gateway_target_id=create_gateway_target_2_response[‘targetId’]

Test gateway with the Strands Agent framework
Test the gateway with the Strands Agents framework to list and call the available tools from MCP server. You can also use other MCP-compatible agents built with different agentic frameworks.

def create_streamable_http_transport():
return streamablehttp_client(
gatewayURL, headers={“Authorization”: f”Bearer {<Bearer_Token>}”}
)
client = MCPClient(create_streamable_http_transport)
with client:
# Call the listTools
tools = client.list_tools_sync()
# Create an Agent with the model and tools
agent = Agent(model=yourModel, tools=tools) ## you can replace with any model you like
# Invoke the agent with the sample prompt. This will only invoke MCP listTools and retrieve the list of tools the LLM has access to. The below does not actually call any tool.
agent(“Hi, can you list all tools available to you”)
# Tool calling
agent(“List all the available pets”)
agent(“Tell me about the pet with petId 3 “)
agent(“When my order will be delivered? My order id is 2”)

You will observe the following output:

I have access to the following tools:
1. **x_amz_bedrock_agentcore_search** – A search tool that returns a trimmed down list of tools based on a provided context/query
2. **api-gateway-target-1___Add_Pet** – Add a new pet to the available Pets
3. **api-gateway-target-1___GetPetById** – Retrieve a specific pet by its ID (requires petId parameter)
4. **api-gateway-target-1___List_Pets** – Retrieves all the available Pets (optional parameters: page, type)
5. **api-gateway-target-2___GetOrderById** – Retrieve a specific order by its ID (requires orderId parameter)
I’ll retrieve all the available pets for you.
Tool #1: api-gateway-target-1___List_Pets
“HTTP/1.1 200 OK”
Here are all the available pets:
1. **Pet ID 1** – Dog – $249.99
2. **Pet ID 2** – Cat – $124.99
3. **Pet ID 3** – Fish – $0.99
I’ll retrieve the details for pet ID 3.
Tool #2: api-gateway-target-1___GetPetById
“HTTP/1.1 200 OK”
Here are the details for pet ID 3:
– **Pet ID**: 3
– **Type**: Fish
– **Price**: $0.99
I’ll check the details of your order with ID 2 to see the delivery information.
Tool #3: api-gateway-target-2___GetOrderById
“HTTP/1.1 200 OK”
Based on your order details:
– **Order ID**: 2
– **Pet Category**: Cat
– **Price**: $124.99
– **Delivery Date**: 02-12-2025 (December 2nd, 2025)
Your cat order will be delivered on **December 2nd, 2025**.

Observability
Enable application logs and tracing for your AgentCore Gateway resource. You will see detailed logs to help monitor and troubleshoot your AgentCore Gateway resource. It will include the tool calls performed by your agentic application, request parameters, responses, and errors if any.
Example logs:

{
“resource_arn”: “arn:aws:bedrock-agentcore:us-west-2:<AWS_Account_Id>:gateway/sample-ac-gateway2-mgtqozexct”,
“event_timestamp”: 1763621922275,
“body”: {
“isError”: false,
“log”: “Executing tool api-gateway-target-1___GetPetById from target W8BCF5VEAZ”,
“id”: “3”
},
“account_id”: “<AWS_Account_Id>”,
“request_id”: “8a70f423-79ee-4168-9d68-b76ad3*****”,
“trace_id”: “324a2ecc08631a55a02bb8f74104****”,
“span_id”: “f58914982450ad9b”,
“timestamp”: “1763621922275”,
“gateway_id”: “sample-ac-gateway2-mgtqozexct”
}
{
“resource_arn”: “arn:aws:bedrock-agentcore:us-west-2: <AWS_Account_Id>:gateway/sample-ac-gateway2-mgtqozexct”,
“event_timestamp”: 1763621922348,
“body”: {
“isError”: false,
“responseBody”: “{jsonrpc=2.0, id=3, result={isError=false, content=[{type=text, text={“id”:3,”type”:”fish”,”price”:0.99}}]}}”,
“log”: “Successfully processed request”,
“id”: “3”
},
“account_id”: “<AWS_Account_Id>”,
“request_id”: “8a70f423-79ee-4168-9d68-b76ad3ef****”,
“trace_id”: “324a2ecc08631a55a02bb8f7410****”,
“span_id”: “f58914982450ad9b”,
“timestamp”: “1763621922348”,
“gateway_id”: “sample-ac-gateway2-mgtqozexct”
}

Along with this, AgentCore Gateway offers detailed CloudWatch metrics including the usage metrics (TargetType, IngressAuthType, EgressAuthType, RequestsPerSession), invocation metrics (Invocations, ConcurrentExecutions, Sessions), performance metrics (Latency, Duration, TargetExecutionTime), and error rates (Throttles, SystemErrors, UserErrors).

AgentCore Gateway also supports AWS X-Ray and OTEL conformant vended spans that customers can use to track invocations across different primitives that are being used.

To learn more, see the AgentCore Gateway Observability documentation.
Clean up
To avoid recurring charges, make sure to delete the resources created by running the following code.

import boto3
gateway_client = boto3.client(‘bedrock-agentcore-control’)
# Deleting the Gateway
Targetresponse = gateway_client.delete_gateway_target( gatewayIdentifier='<Gateway_Id>’, targetId='<Target_Id>’)print(response)
# Deleting the Gateway
response = gateway_client.delete_gateway(
gatewayIdentifier='<Gateway_Id>’)
print(response)

Conclusion
AgentCore Gateway now supports Amazon API Gateway as a target, exposing REST APIs as MCP-compatible endpoints. You can bring your existing API infrastructure to agentic use cases while using your current security and observability tools.
Visit our developer documentation and workshop to learn more and get started today.

About the authors
With over 6+ years at AWS, Sparsh Wadhwa brings deep expertise in serverless, event-driven architectures, and Generative AI to his work with ISV customers in India. As a Solutions Architect, he partners with Independent Software Vendors to reimagine their products for the cloud era—from modernizing legacy systems to embedding AI capabilities that differentiate their offerings. Sparsh believes the best solutions emerge from understanding both technical possibilities and business context.
Heeki Park is a Principal Solutions Architect at AWS. In his 9+ years at AWS, he helped enterprise customers think about how to build and operate cloud-native applications, adopt serverless and event-driven patterns, and build pragmatic generative AI applications. Heeki is an avid runner and enjoys analyzing activity data to measure improvement in cardiovascular fitness.
Dhawal Patel is a Principal Generative AI Tech lead at AWS. He has worked with organizations ranging from large enterprises to mid-sized startups on problems related to agentic AI, deep learning, and distributed computing.

Cisco and Splunk have introduced the Cisco Time Series Model, a univariate zero shot time series foundation model designed for observability and security metrics. It is released as an open weight checkpoint on Hugging Face under an Apache 2.0 license, and it targets forecasting workloads without task specific fine tuning. The model extends TimesFM 2.0 with an explicit multiresolution architecture that fuses coarse and fine history in one context window.

https://arxiv.org/pdf/2511.19841

Why observability needs multiresolution context?

Production metrics are not simple single scale signals. Weekly patterns, long term growth and saturation are visible only at coarse resolutions. Saturation events, traffic spikes and incident dynamics show up at 1 minute or 5 minute resolution. The common time series foundation models work at a single resolution with context windows between 512 and 4096 points, while TimesFM 2.5 extends this to 16384 points. For 1 minute data this still covers at most a couple of weeks and often less.

This is a problem in observability where data platforms often retain only old data in aggregated form. Fine grained samples expire and survive only as 1 hour rollups. Cisco Time Series Model is built for this storage pattern. It treats coarse history as a first class input that improves forecasts at the fine resolution. The architecture operates directly on a multiresolution context instead of pretending that all inputs live on a single grid.

https://arxiv.org/pdf/2511.19841

Multiresolution input and forecasting objective

Formally, the model consumes a pair of contexts, (xc, xf). The coarse context (x_c) and the fine context (x_f) each have length up to 512. The spacing of (xc) is fixed at 60 times the spacing of (xf). A typical observability setup uses 512 hours of 1 hour aggregates and 512 minutes of 1 minute values. Both series terminate at the same forecast cut point. The model predicts a horizon of 128 points at the fine resolution, with a mean and a set of quantiles from 0.1 to 0.9.

Architecture, TimesFM core with resolution embeddings

Internally, Cisco Time Series Model reuses the TimesFM patch based decoder stack. The inputs are normalized, patched into non overlapping chunks, and passed through a residual embedding block. The transformer core consists of 50 decoder only layers. A final residual block maps tokens back to the horizon. The research team remove positional embeddings and instead rely on patch ordering, the multiresolution structure and a new resolution embedding to encode structure.

Two additions make the architecture multiresolution aware. A special token, often called ST in the report, is inserted between the coarse and fine token streams. It lives in sequence space and marks the boundary between resolutions. Resolution embeddings, often called RE, are added in model space. One embedding vector is used for all coarse tokens and another for all fine tokens. Ablation studies in the paper show that both components improve quality, especially in long context scenarios.

The decode procedure is also multiresolution. The model outputs mean and quantile forecasts for the fine resolution horizon. During long horizon decoding, newly predicted fine points are appended to the fine context. Aggregates of these predictions update the coarse context. This creates an autoregressive loop in which both resolutions evolve together during forecasting.

https://arxiv.org/pdf/2511.19841

Training data and recipe

Cisco Time Series Model is trained by continued pretraining on top of TimesFM weights. The final model has 500 million parameters. Training uses AdamW for biases, norms and embeddings, and Muon for the hidden layers, with cosine learning rate schedules. The loss combines mean squared error on the mean forecast with quantile loss over the quantiles from 0.1 to 0.9. The team trains for 20 epochs and picks the best checkpoint by validation loss.

The dataset is large and skewed toward observability. The Splunk team reports about 400 million metrics time series from their own Splunk Observability Cloud deployments, collected at 1 minute resolution over 13 months and partly aggregated to 5 minute resolution. The research team states that the final corpus contains more than 300 billion unique data points, with about 35 percent 1 minute observability, 16.5 percent 5 minute observability, 29.5 percent GIFT Eval pretraining data, 4.5 percent Chronos datasets and 14.5 percent synthetic KernelSynth series.

Benchmark results on observability and GIFT Eval

The research team evaluate the model on two main benchmarks. The first is an observability dataset derived from Splunk metrics at 1 minute and 5 minute resolution. The second is a filtered version of GIFT Eval, where datasets that leak TimesFM 2.0 training data are removed.

On observability data at 1 minute resolution with 512 fine steps, Cisco Time Series Model using a 512 multiresolution context reduces mean absolute error from 0.6265 for TimesFM 2.5 and 0.6315 for TimesFM 2.0 to 0.4788, with similar improvements in mean absolute scaled error and continuous ranked probability score. Similar gains appear at 5 minute resolution. Across both resolutions, the model outperforms Chronos 2, Chronos Bolt, Toto and AutoARIMA baselines under the normalized metrics used in the paper.

On the filtered GIFT Eval benchmark, Cisco Time Series Model matches the base TimesFM 2.0 model and performs competitively with TimesFM-2.5, Chronos-2 and Toto. The key claim is not universal dominance but preservation of general forecasting quality while adding a strong advantage on long context windows and observability workloads.

https://arxiv.org/pdf/2511.19841

Key Takeaways

Cisco Time Series Model is a univariate zero shot time series foundation model that extends the TimesFM 2.0 decoder only backbone with a multiresolution architecture for observability and security metrics.

The model consumes a multiresolution context, with a coarse series and a fine series, each up to 512 steps long, where the coarse resolution is 60 times the fine resolution, and it predicts 128 fine resolution steps with mean and quantile outputs.

Cisco Time Series Model is trained on more than 300B data points, with more than half from observability, mixing Splunk machine data, GIFT Eval, Chronos datasets and synthetic KernelSynth series, and it has about 0.5B parameters.

On observability benchmarks at 1 minute and 5 minute resolutions, the model achieves lower error than TimesFM 2.0’s, Chronos and other baselines, while retaining competitive performance on the general purpose GIFT Eval benchmark.

Check out the Paper, Blog and Model Card on HF. Feel free to check out our GitHub Page for Tutorials, Codes and Notebooks. Also, feel free to follow us on Twitter and don’t forget to join our 100k+ ML SubReddit and Subscribe to our Newsletter. Wait! are you on telegram? now you can join us on telegram as well.
The post Cisco Released Cisco Time Series Model: Their First Open-Weights Foundation Model based on Decoder-only Transformer Architecture appeared first on MarkTechPost.