Why Your Black Friday Campaigns Need a ‘Bounce-Back’ Strategy This …

Ah, Black Friday. The unofficial Hunger Games of ecommerce. Brands throw down, customers swipe up, and everyone’s cart is a battlefield of BOGO deals and last-minute splurges. 

But here’s the thing. While Black Friday (and its entourage of early deals) brings a rush of revenue, the aftermath can feel like crickets. Post-holiday slump, anyone?

The reality? Black Friday isn’t just a day anymore…it’s a whole season! 

Our data shows jumps in customer activity even the week before Black Friday, thanks to brands kicking off their promos earlier each year. 

But once the last “Doorbuster Alert!” email is sent, most brands go quiet and that’s a problem. Why? Because your Black Friday customers are still primed to shop!

This is where a bounce-back strategy comes in to save the day (and your Q4 revenue goals). 

Simply put, a bounce-back strategy re-engages your Black Friday shoppers, turning one-time buyers into repeat customers. It’s about riding that wave of holiday shopping enthusiasm, not letting it crash and burn.

And here’s the best part – customers who make a second purchase shortly after their first are significantly more likely to become long-term patrons. Specifically, after a second purchase, there’s a 49% chance they’ll buy again, and after a third purchase, the likelihood increases to 62%. 

That means not just more revenue now, but a chance to build loyalty that pays off well beyond the holiday season.

So, are you ready to bounce back or are you leaving money on the table? 

What Is a Bounce-Back Strategy and Why Does It Work?

Have you ever been in this situation? You’ve just devoured the best meal of your life and the waiter swings by with a dessert tray. 

You’re full, sure but who can say no to a perfectly plated chocolate lava cake? Of course you order it!

That’s the magic of good timing and it’s exactly what a bounce-back strategy does for your ecommerce business.

A bounce-back strategy is all about re-engaging customers while they’re still in the zone. 

And that zone is certainly no bigger than the holiday season. When someone makes a purchase during Black Friday, they’re in peak shopping mode…primed, ready, and still riding that dopamine high from scoring deals. 

Why let that momentum fade? 

With targeted ads or emails that hit shortly after their initial buy, you can turn one-time shoppers into repeat customers.

Here’s where things get tricky thought. Tools like Klaviyo can identify some of your recent buyers, but not all. Why? Klaviyo relies on cookies to track visitors and those cookies expire after seven days. If a customer revisits your site on day eight or later, they might as well be a ghost to Klaviyo.

That’s where Customers.ai Signal comes in. We go beyond cookie tracking to identify returning visitors (whether they’re browsing anonymously or logged in). 

By enriching visitor data in real time, Customers.ai signals platforms like Klaviyo or Shopify that this is a returning customer. This gives you the power to hit them with the right message at the right time—a bounce-back offer tailored to their browsing behavior or past purchases.

Think of it as turning what would have been a missed connection into a perfectly timed “welcome back” moment. And during the holiday frenzy, when every second counts, that kind of precision can make all the difference.

Timing Is Everything: When to Launch Your Bounce-Back Campaign

Timing is everything when it comes to bounce-back campaigns. Hit your customers too soon, and they might feel overwhelmed. Wait too long, and the spark of their initial purchase has fizzled out. 

The sweet spot? Typically within 3–7 days of their first buy.

Here’s why this window works. Post-purchase, shoppers are still in what we like to call the holiday shopping glow. 

They’re engaged, your brand is fresh in their minds, and they’re more likely to act on an offer that feels like a continuation of their shopping journey. 

This is especially true during the Black Friday-to-Cyber Monday frenzy when customers are conditioned to expect deals.

Holiday Momentum: Use It or Lose It

Think of the holiday shopping season like a snowball rolling downhill. The momentum is powerful but it doesn’t last forever. 

Customers who bought during Black Friday are primed to keep spending, especially if you give them the right nudge. Perhaps a limited-time discount, a personalized product recommendation, or free shipping on their next order. 

Bounce-back campaigns are your way to capitalize on that momentum before it melts away.

Timing Based on Product Types

Remember, timing is everything and that means the timing should be specific to the individual action. Your bounce-back strategy should reflect that.

Consumables or fast-moving goods

Launch bounce-back campaigns within 1–3 days. Think snacks, beauty products, or holiday-themed items where a sense of urgency drives quick decisions.

Example:

Product: Holiday-themed coffee blends.

Ad Copy: “Still craving holiday cheer? Reorder your favorite holiday blend now and get 10% off your next bag—offer ends in 24 hours!”

Visuals: A cozy image of the coffee being brewed with a festive backdrop and a countdown timer emphasizing the urgency.

Why it works: Consumables are often purchased on impulse and shoppers may already be thinking about replenishing or trying another variation. The quick follow-up keeps the product top of mind.

Apparel and mid-tier products

The 3–5 day range works best here, giving your customers time to enjoy their first purchase while enticing them with complementary items.

Example:

Product: Winter jackets.

Ad Copy: “Love your new jacket? Complete the look with our matching scarf—20% off when you shop within the next 48 hours!”

Visuals: A carousel ad showing the jacket styled with complementary items, such as scarves, hats, and gloves.

Why it works: Customers have had time to receive or try out their purchase, making this the perfect opportunity to suggest add-ons or related items.

Luxury or high-consideration items

Wait 5–7 days, offering an exclusive follow-up deal or showcasing related high-value products to reinforce their investment in your brand.

Example:

Product: High-end noise-canceling headphones.

Ad Copy: “Enhance your listening experience. Get $50 off our premium protective case and charging dock when you shop this week.”

Visuals: A sleek product shot of the headphones alongside the case and dock, with a tagline like, “Protect your investment in style.”

Why it works: Luxury purchases often involve more deliberation. A follow-up after several days allows customers to appreciate their initial investment and consider accessories or related products that enhance the value.

Pro Tip: Create a Sense of Urgency

Time-sensitive language like “Your exclusive deal expires in 48 hours!” or “Complete your holiday look before it’s gone!” helps nudge customers to act. 

Pair this with targeted ads and emails that align with their browsing or buying behavior, and you’ve got a winning formula.

By nailing the timing of your bounce-back campaigns, you’re not just driving repeat purchases, you’re reinforcing customer loyalty during the most competitive shopping season of the year.

Crafting the Perfect Bounce-Back Offer

A bounce-back offer isn’t just about throwing out a random discount. It’s about crafting an irresistible deal that feels like a natural continuation of the customer’s journey. 

By tailoring offers to their previous purchases and delivering them through strategic channels like ads and emails, you can keep your brand top of mind and drive repeat sales. 

Let’s break it down.

1. Discounts on Complementary Products

Why it works: Customers often need accessories or add-ons to complete their purchase. Offering these at a discount encourages a follow-up sale while reinforcing the value of their initial buy.

Ads:

Example: If they bought a fitness tracker, show them a carousel ad with discounted bands or a charging dock.

Ad Copy: “Gear up! Get 20% off accessories for your fitness tracker—offer ends in 48 hours!”

Emails:

Example: A personalized email recommending matching products.

Subject Line: “Love your new tracker? Complete the set!”

Body: Showcase the complementary products with a limited-time discount to create urgency.

2. Exclusive Loyalty Rewards

Why it works: Customers love to feel valued. Offering loyalty perks as part of your bounce-back strategy not only drives sales but also builds long-term brand affinity.

Ads:

Example: Promote a special “holiday bonus” for signing up for your loyalty program or purchasing again.

Ad Copy: “Double your points this week only! Earn rewards faster when you shop again by [DATE].”

Emails:

Example: Reward repeat shoppers with early access to new products or special VIP pricing.

Subject Line: “You’re in the club! Exclusive rewards just for you.”

Body: Tease upcoming perks and offer a bounce-back incentive like bonus points or member-exclusive discounts.

3. Limited-Time Free Shipping Deals

Why it works: Free shipping is often the nudge customers need to convert, especially for smaller follow-up purchases.

Ads:

Example: Run a retargeting ad for customers who abandoned their cart or purchased a lightweight item.

Ad Copy: “Forgot something? Come back and get free shipping—today only!”

Emails:

Example: Highlight free shipping for purchases made within a short time frame.

Subject Line: “Back for more? Enjoy free shipping!”

Body: Remind them of their recent purchase and offer free shipping as an added incentive to return.

Examples of Offers That Work Well Post-Holiday

Bundle Deals: “Bundle up! Get 15% off when you buy a matching set.”

Gift-With-Purchase: “Enjoy a free travel pouch with your next purchase—only for a limited time!”

Limited-Time Discounts: “You’re part of our VIP crowd—grab 25% off your next purchase by [DATE].”

Product Recommendations: “Customers who bought [Product A] also loved [Product B]—grab it now with an exclusive discount!”

Don’t Forget: Pair Ads and Emails for Maximum Impact

Don’t choose between ads or emails…combine them! 

Retarget customers with ads on platforms like Facebook and Instagram while reinforcing the same message in their inbox. This dual approach ensures they’ll see the offer wherever they spend their time online.

Section 4: How to Use Customers.ai to Execute Your Bounce-Back Campaign

Audience segmentation:

Retarget first-time buyers from Black Friday/Cyber Monday.

Send to Meta Ads

Create new flows in Klaviyo

Use Lookalike Audiences to scale.

Creative ideas:

Carousel ads showcasing complementary items.

“Thank you” ads with a personal touch.

2 creative email ideas

Ad formats and placements: Which work best for bounce-back campaigns.

How to Use Customers.ai to Execute Your Bounce-Back Campaign

Crafting the perfect bounce-back offer is just the start. The good stuff happens when you use Customers.ai to ensure that offer lands in front of the right people, at the right time, on the right platforms. 

Here’s how to leverage Customers.ai to supercharge your bounce-back campaigns with precision and creativity.

Audience Segmentation: Targeting with Laser Precision

1. Retarget First-Time Buyers from Black Friday/Cyber Monday

Use Customers.ai to identify your Black Friday buyers and categorize them into segments based on purchase data. Did they buy small-ticket items? Big-ticket items? Create tailored follow-ups for each group.

2. Send Data to Meta Ads for Retargeting

Push segmented audiences directly to Meta Ads. For example, retarget recent buyers with complementary products or time-sensitive offers like discounts or free shipping.

3. Create New Flows in Klaviyo

Signal return visits captured by Customers.ai to Klaviyo, and set up personalized email flows. For example:

Flow 1: “Thanks for shopping!” email → Follow-up offer after 2 days.

Flow 2: “We see you browsing!” email → Incentive for returning visitors captured by Customers.ai.

4. Use Lookalike Audiences to Scale

Build Lookalike Audiences based on your Black Friday buyers. Use Customers.ai’s enriched data to target new shoppers with behaviors similar to your existing audience.

Creative Ideas: Engage with Stand-Out Messaging

Carousel Ads Showcasing Complementary Items

Example: For customers who bought a winter coat, display a carousel ad with matching scarves, gloves, and boots.

Ad Copy: “Complete the look! Pair your new coat with these winter must-haves. Shop now and save 15%!”

“Thank You” Ads with a Personal Touch

Example: A static image or short video ad that thanks customers for their purchase while highlighting a special bounce-back offer.

Ad Copy: “Thanks for shopping with us this Black Friday! Here’s 10% off your next order—just for being awesome.”

Creative Email Ideas

Email 1: The Personalized Recommendation

Subject Line: “Loved [Product Name]? You’ll adore these too!”

Body: Showcase related products, add an exclusive discount, and include a time-sensitive call-to-action like “Offer ends in 48 hours!”

Email 2: The Holiday Countdown

Subject Line: “Time’s running out for our holiday deals!”

Body: Highlight the urgency with a bold countdown timer and emphasize the value of your bounce-back offer. For example, “Your free shipping deal ends in 12 hours!”

Ad Formats and Placements: What Works Best for Bounce-Back Campaigns

Ad Formats:

Carousel Ads: Ideal for showcasing multiple complementary products or bundles.

Dynamic Product Ads: Automatically display the exact items customers browsed or purchased, keeping your message relevant.

Video Ads: A short thank-you message or product highlight can create a more personal connection.

Placements:

Instagram Stories and Reels: Perfect for eye-catching, time-sensitive offers that leverage interactive elements like polls or countdowns.

Facebook Feed and Marketplace: Great for detailed product showcases and more deliberate shopping behavior.

Google Display Network: Reengage your audience across other websites they visit with visually engaging banner ads.

By combining the advanced audience segmentation capabilities of Customers.ai with creative, well-timed ads and emails, your bounce-back campaigns can go beyond driving repeat purchases and build long-term loyalty.

Don’t Let the Holiday Momentum Fizzle Out

Black Friday might bring the fireworks but without a bounce-back strategy, you’re missing the afterparty. 

Your customers just made a purchase and they’re still riding that shopping high! Don’t let it fade into oblivion. 

This is your moment to stay in their orbit, turn one-time buyers into repeat customers, and build the kind of loyalty that pays off big time.

A killer bounce-back strategy doesn’t just happen by accident. You need the right timing, the perfect offer, and tools like Customers.ai to do the heavy lifting—tracking return visitors, syncing data with Klaviyo, and making sure every ad and email lands like it was custom-built for each shopper (spoiler: it was).

So, what’s the move? 

Don’t wait for next year’s resolutions. Start planning your bounce-back strategy now, while your Black Friday buyers are still fresh in the game. The post-holiday slump is real but it doesn’t have to be your reality.

Ready to bounce back like a boss? Customers.ai has your back. Try it free and re-engage your Black Friday shoppers with ease!

See Who Is On Your Site Right Now!

Get names, emails, phone numbers & more.

Try it Free, No Credit Card Required

Start Your Free Trial

The post Why Your Black Friday Campaigns Need a ‘Bounce-Back’ Strategy This Holiday Season appeared first on Customers.ai.

Visitor Tracking Tools: The 25 Best Solutions for 2025 [Reviewed]

Here’s a wild stat to kick things off: 91% of users won’t return to a website after a bad experience. That’s a LOT of missed opportunities, especially when you could be using tools to understand what went wrong and fix it before it happens again.

Tools like…visitor tracking? Exactly!

Visitor tracking tools have come a long way. They’re no longer limited to basic metrics like page views or bounce rates. Today’s platforms dive deeper, going beyond the metrics and uncovering the “why” behind user actions.

But boy are there a lot of tools out there and it can be pretty overwhelming trying to figure which ones are right for you. 

That’s what we’re here for. We’re skipping the fluff and diving into the tools that bring real value to the table. Whether you’re optimizing an ecommerce store, generating B2B leads, or driving traffic to your latest campaign, these tools help you take control of your site’s performance. 

Let’s get into it.

Unlock High-Intent Leads Hiding on Your Site

Book a demo of Customers.ai’s U.S. website visitor identification, customer journey insights and remarketing platform to skyrocket conversions and sales.

Book a Demo

Beyond the Basics: Features That Make Modern Visitor Tracking Tools Stand Out

Gone are the days when visitor tracking was just about counting clicks and logging time on page. Today’s tools are packed with features that give you a front-row seat to what your visitors are really up to. 

Let’s break down some of the capabilities redefining what’s possible for marketers:

1. Real-Time User Behavior Visualization

Think of this as a live feed of your website in action. With real-time user tracking, you can watch your visitors’ behavior as it unfolds – clicks, scrolls, hovers, and all.

How awesome, right? With real-time user behavior visualization, you can:

Spot bottlenecks instantly and adjust on the fly.

See where users linger and where they drop off.

Identify unexpected navigation paths that reveal how users actually explore your site.

Catch technical issues, like broken links or slow-loading pages, before they impact more users.

Monitor real-time conversion funnels to optimize your checkout or sign-up process on the spot.

It’s like having X-ray vision for your site, except way less creepy and entirely actionable.

2. Predictive Analytics: Anticipate Their Next Move

Ever wish you could predict what your visitors will do next? Predictive analytics gets you pretty close. By analyzing past behavior and patterns, these tools can:

Identify users who are likely to bounce so you can trigger re-engagement tactics.

Suggest upsell or cross-sell opportunities based on real-time intent signals.

Forecast demand trends to adjust inventory, content, or offers in advance, ensuring you’re always a step ahead of user needs.

Basically, you’re not just reacting to what’s happening—you’re staying one step ahead.

3. AI-Driven Insights for Personalization

Let’s talk about the power of AI. Modern tools don’t just collect data, they analyze it and serve up actionable insights. Think personalized experiences on steroids:

AI identifies user preferences and suggests tailored content or offers.

It spots trends that would take a human hours (or days) to detect.

Automatically segment audiences into micro-groups based on behavior, ensuring hyper-relevant campaigns and messaging for every type of visitor.

These three capabilities aren’t just bells and whistles. They’re features designed to make your life easier and your results stronger. Modern visitor tracking tools aren’t playing around and neither should you.

AI-Powered Advertising

How to Unlock AI and Lead Capture Tech for 10X Return on Ad Spend

HOSTED BY

Larry Kim

Founder and CEO, Customers.ai

Free Webinar: Watch Now

Integrating Visitor Tracking with Your Marketing Stack: Where the Magic Happens

No tool works in a vacuum, especially when it comes to ecommerce. Visitor tracking tools are great on their own, but when you integrate them with the rest of your marketing stack, that’s when things get really interesting.

We’re talking about smarter campaigns, sharper targeting, and insights so good they’ll make you wonder how you ever lived without them. 

Whether it’s powering your CRM, supercharging your automations, or fueling your A/B tests, visitor tracking tools play a huge role.

Here’s how these integrations turn a good strategy into an unstoppable one.

1. CRM Systems: Build Customer Profiles That Mean Business

A CRM is only as effective as the data it’s built on. Integrating visitor tracking tools enhances your CRM by providing real-time insights into what your customers are doing on your site, giving you the information you need to engage them more effectively.

For instance, when a visitor spends 10 minutes browsing your premium sneakers category, your CRM doesn’t just note that – they’re flagged for follow-up with tailored campaigns featuring similar products. 

The result? 

Smarter segmentation, more relevant outreach, and a serious boost in conversions.

2. Marketing Automation: Set It and Sell It

Let’s face it, automated campaigns need data to thrive and integrating visitor tracking tools adds the context you need to create campaigns that perform.

For example, abandoned cart emails generate 320% more revenue when they’re automated and personalized. 

With visitor tracking, that email doesn’t just say, “Hey, you forgot something.” It reminds them exactly which item they left behind and throws in a discount code to nudge them across the finish line. 

It’s real-time relevance and it works!

3. A/B Testing Tools: No More Guessing Games

What’s better than running an A/B test? Running one with insights that take the guesswork out of the equation. 

Visitor tracking tools show you what users are doing and they help you understand why, making your tests more informed and your results more impactful.

Take a split test on two landing pages. 

Your tracking tool reveals that users on Version A hesitate right after viewing the price, while Version B keeps them clicking straight to checkout.

 Armed with that data, you don’t just declare a winner, you learn what works and apply it everywhere.

When your visitor tracking tools are working seamlessly with your marketing stack, it’s not just about gathering data—it’s about transforming it into decisions that make your ecommerce sales soar.

Choosing the Right Visitor Tracking Tools for Your Store

Picking the right visitor tracking tool isn’t about finding the flashiest features. It’s about finding the one that works for you. 

For ecommerce marketers, that means a tool that doesn’t just track clicks but actually helps you drive revenue.

Whether you’re focused on optimizing product pages, reducing cart abandonment, or scaling your marketing campaigns, the right tool can make all the difference. 

But with so many options out there, it’s easy to feel overwhelmed. Let’s cut through the noise with a straightforward approach to choosing the perfect fit.

1. Get Clear on Your Goals

Before diving into features and pricing, think about what you actually need the tool to do. 

Are you focused on tracking visitor behavior across your ecommerce funnel? 

Do you need real-time insights to catch issues as they happen? 

Or maybe it’s about integrating seamlessly with your existing marketing stack.

For example, if reducing cart abandonment is your top priority, look for tools with session replay or heatmap functionality to pinpoint where shoppers drop off. 

Clarity on your goals means no wasted features and no wasted money.

2. Check for Scalability

Your business today isn’t the same as your business next year. 

Can the tool grow with you? 

Look for platforms that can handle increased traffic, offer advanced features you might need down the road, and provide flexibility as your marketing strategies evolve.

For ecommerce marketers, scalability is a must. You don’t want to switch tools during your busiest sales season because your current one can’t keep up.

3. Test Integrations Like Your Business Depends on It (Because It Does)

If your visitor tracking tool doesn’t play well with your CRM, email marketing, or other essential systems, you’re leaving money on the table. 

Before committing, ensure the tool integrates seamlessly with the platforms you already use, whether that’s Shopify, Klaviyo, or Google Analytics.

Pro tip: Check if the tool offers API support for custom integrations. The more connected your data, the better your insights and your results.

4. Prioritize User-Friendliness

You don’t need a PhD in data science to use a visitor tracking tool (or at least, you shouldn’t). 

Choose a platform that gives you powerful insights but is intuitive enough for you and your team to use without endless training.

A simple setup, clean dashboards, and clear reporting are key. If you’re spending more time learning the tool than acting on its data, it’s not the right fit.

5. Take It for a Test Drive

Most tools offer free trials or demos. Use them! During the trial, focus on how well the tool aligns with your day-to-day needs:

Can you easily find the insights that matter?

Does it help solve the problems you identified in your goals?

Is customer support responsive and helpful?

This trial period is your chance to see how the tool performs under real-world conditions, so don’t skip it.

As you evaluate your tools, it’s important to remember that the right visitor tracking tool isn’t about finding what’s popular, it’s about finding what delivers results for your ecommerce business.

With a little planning and a clear focus on your goals, you’ll find a tool that not only tracks visitors but helps you turn those visitors into revenue.

25 Visitor Tracking Tools (with Features, Ratings & Reviews)

Below, you’ll find a curated list of the top 25 tools, complete with their standout features, pricing, ratings, and reviews. Whether you’re focused on optimizing your ecommerce funnel, improving user experience, or boosting conversions, there’s something here for every business. 

Customers.ai Website Visitor Identification

Customers.ai Customer Journey Mapping Tool

Customers.ai Signal

Customers.ai Visitor Demographics Tool

Google Analytics 4 (GA4)

Hotjar

Crazy Egg

Adobe Analytics

Piwik PRO

Mixpanel

FullStory

Lucky Orange

Woopra

Smartlook

Heap

Mouseflow

Kissmetrics

Aloware

Albacross

HubSpot Analytics

Salesforce Marketing Cloud Interaction Studio

Zoho PageSense

Segment

VWO (Visual Website Optimizer)

Chartbeat

1. Website Visitor Identification & Tracking Pixel

Best for: Businesses aiming to identify and engage anonymous website visitors.

What It Does: The Customers.ai Website Visitor Identificaton tool reveals the identities of anonymous website visitors, providing details like company names, contact information, and browsing behavior. It’s ideal for B2B and ecommerce brands looking to connect with high-value prospects.

Standout Features:

Real-time visitor identification

Detailed visitor profiles

Seamless CRM integration

Pricing: Starts at $99/month, scalable based on traffic and features.

Rating: ★★★★★ (4.8/5)

Reviews:

“Setup was quick, and insights started flowing immediately.” – G2 Reviewer

“Converted several anonymous visitors into leads; a game-changer for our sales team.” – Capterra Reviewer

2. Customer Journey Mapping & Visitor Tracking Tool

Best for: Marketers seeking to visualize and optimize the customer journey.

What It Does: The Customers.ai customer journey mapping tool allows businesses to map out and analyze the paths visitors take on their website, identifying drop-off points and opportunities for improvement.

Standout Features:

Comprehensive journey visualization

Behavioral analytics

Integration with marketing automation platforms

Pricing: Included in Customers.ai’s suite.

Rating: ★★★★★ (4.8/5)

Reviews:

“Helped us pinpoint where customers were dropping off, allowing for targeted optimizations.” – Trustpilot Reviewer

3. Signal Return Visitor Identification & Tracking

Best for: Teams needing real-time alerts on significant return visitor activities.

What It Does: Signal by Customers.ai is a tool designed to enhance Klaviyo’s tracking capabilities by identifying return visitors that Klaviyo may miss due to its 7-day cookie expiration. Signal helps you capture and engage with high-intent visitors who are in-market and ready to buy.

Standout Features:

Customizable alert settings

Integration with communication tools

Real-time data processing

Pricing: Part of Customers.ai’s platform; contact for details.

Rating: ★★★★★ (4.8/5)

Reviews:

“The real-time alerts have significantly improved our response times to potential leads.” – G2 Reviewer

4. Visitor Tracking Demographics Tool

Best for: Businesses aiming to understand the demographic profiles of their visitors.

What It Does: The visitor tracking and demographics tool provides insights into visitor demographics, including age, gender, location, and interests, aiding in targeted marketing strategies.

Standout Features:

Detailed demographic breakdowns

Integration with advertising platforms

Customizable reporting

Pricing: Included in Customers.ai’s offerings

Rating: ★★★★★ (4.8/5)

Reviews:

“Understanding our visitor demographics has allowed for more personalized marketing campaigns.” – Capterra Reviewer

5. Google Analytics 4 (GA4) Visitor Tracking

Best for: Businesses of all sizes seeking comprehensive web analytics.

What It Does: GA4 offers in-depth insights into website traffic and user behavior, supporting data-driven decisions.

Standout Features:

Event-driven data model

Cross-platform tracking

Enhanced privacy controls

Pricing: Free

Rating: ★★★★☆ (4.7/5)

Reviews:

“A robust tool for tracking and analyzing website performance.” – G2 Reviewer

6. Hotjar Visitor Tracking & Heatmaps

Best for: Businesses looking to understand user behavior through heatmaps and session recordings.

What It Does: Hotjar provides visual representations of user interactions, highlighting areas of interest and potential issues.

Standout Features:

Heatmaps

Session recordings

User feedback polls

Pricing: Free plan available; paid plans start at $39/month.

Rating: ★★★★☆ (4.6/5)

Reviews:

“Invaluable for understanding how users interact with our site.” – Trustpilot Reviewer

7. Crazy Egg Heatmaps & Visitor Tracking

Best for: Marketers seeking visual insights into user behavior.

What It Does: Crazy Egg offers heatmaps, scrollmaps, and A/B testing to optimize website performance.

Standout Features:

Heatmaps and scrollmaps

A/B testing

User recordings

Pricing: Plans start at $24/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“The visual reports have been crucial in improving our site’s user experience.” – Capterra Reviewer

8. Adobe Analytics Visitor Tracking

Best for: Enterprises needing advanced analytics capabilities.

What It Does: Adobe Analytics provides deep insights into customer journeys across multiple channels.

Standout Features:

Cross-channel data integration

Predictive analytics

Customizable reporting

Pricing: Custom pricing; contact Adobe for details.

Rating: ★★★★☆ (4.4/5)

Reviews:

“A powerful tool for comprehensive analytics across our digital platforms.” – G2 Reviewer

9. Piwik PRO Analytics Tracking

Best for: Organizations prioritizing data privacy and compliance, such as those in healthcare, finance, and government sectors.

What It Does: Piwik PRO is an analytics suite that offers comprehensive web and app analytics with a strong focus on user privacy and data security. It enables businesses to track user behavior while ensuring compliance with data protection regulations like GDPR.

Standout Features:

Privacy Compliance: Built-in features to help comply with GDPR, HIPAA, and other data protection laws.

On-Premises and Cloud Hosting: Offers flexible hosting options to meet various security requirements.

User-Level Data Access: Provides detailed insights into individual user journeys.

Tag Manager Integration: Simplifies the deployment and management of tracking codes.

Pricing: Custom pricing based on business needs; contact Piwik PRO for a quote.

Rating: ★★★★☆ (4.5/5)

Reviews:

Data Privacy: “Piwik PRO’s commitment to data privacy is unparalleled, making it our go-to analytics tool.” – G2 Reviewer

Customization: “The platform’s flexibility allows us to tailor analytics to our specific requirements.” – Trustpilot Reviewer

Support: “Their customer support team is responsive and knowledgeable, assisting us through complex setups.” – Capterra Reviewer

10. Mixpanel Advanced Analytics & Tracking Tool

Best for: Product teams aiming to understand user behavior and drive engagement.

What It Does: Mixpanel offers advanced analytics to track user interactions with your website or app, helping identify trends and optimize user experiences.

Standout Features:

Advanced Funnel Analysis: Pinpoint where users drop off in their journey.

Cohort Analysis: Monitor user groups over time to understand retention.

Customizable Dashboards: Tailor insights to your specific business needs.

Pricing: Free plan available; paid plans start at $25/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“The funnel insights helped us improve our signup flow significantly.” – G2 Reviewer

11. FullStory Session Replay & Visitor Tracking Tool

Best for: Teams focused on enhancing user experience through session replays and analytics.

What It Does: FullStory captures every user interaction on your site, offering session replays, heatmaps, and analytics to help improve user experience.

Standout Features:

Session Replays: Watch exactly how users interact with your site.

Heatmaps: Identify areas of high engagement or confusion.

Error Tracking: Detect and prioritize user-impacting bugs.

Pricing: Custom pricing; free trial available.

Rating: ★★★★☆ (4.7/5)

Reviews:

“FullStory showed us exactly where users were struggling on our checkout page.” – Trustpilot Reviewer

12. Lucky Orange Visitor Tracking Tool

Best for: Small to medium-sized businesses seeking affordable UX insights.

What It Does: Lucky Orange combines session replays, heatmaps, and live chat to provide a holistic view of visitor behavior.

Standout Features:

Dynamic Heatmaps: See clicks, scrolls, and mouse movements.

Live Chat Integration: Engage visitors in real time.

Conversion Funnels: Understand drop-off points in user journeys.

Pricing: Plans start at $18/month.

Rating: ★★★★☆ (4.6/5)

Reviews:

“An affordable way to get detailed insights into user behavior.” – G2 Reviewer

13. Woopra User Journey & Visitor Tracking 

Best for: Businesses aiming for real-time customer journey analytics.

What It Does: Woopra provides end-to-end tracking of user journeys, offering insights to optimize customer experiences and increase retention.

Standout Features:

Real-Time Analytics: Monitor user activity as it happens.

Customer Journeys: Visualize and analyze the entire user lifecycle.

Integrations: Connect with various tools like Salesforce, Slack, and more.

Pricing: Free plan available; paid plans start at $349/month.

Rating: ★★★★☆ (4.4/5)

Reviews:

“Woopra’s real-time insights have been crucial for our marketing strategies.” – Capterra Reviewer

14. Smartlook Visitor Tracking & Session Analysis

Best for: Companies needing detailed user behavior insights for websites and mobile apps.

What It Does: Smartlook records user sessions and provides detailed analytics to help businesses understand how visitors interact with their sites or apps.

Standout Features:

Mobile App Tracking: Optimize user experiences on apps.

Automatic Event Tracking: Capture interactions without manual setup.

Customizable Dashboards: Focus on the metrics that matter most.

Pricing: Free plan available; paid plans start at $39/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Smartlook has been a lifesaver for improving app usability.” – Capterra Reviewer

15. Heap Analytics & Visitor Tracking Tools

Best for: Data-driven teams seeking automated analytics.

What It Does: Heap automatically captures every user interaction on your website or app, removing the need for manual event tracking.

Standout Features:

Automatic Data Capture: No manual tracking setup required.

Path Analysis: Visualize user flows through your site or app.

Retention Reports: Identify what keeps users coming back.

Pricing: Free plan available; paid plans start at $3,600/year.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Heap’s automatic tracking saved us countless hours of setup work.” – G2 Reviewer

16. Mouseflow User Session & Visitor Tracking

Best for: Businesses aiming to enhance user experience through session recordings and heatmaps.

What It Does: Mouseflow captures user sessions, providing insights into behavior patterns to optimize website performance.

Standout Features:

Session Replay: Watch real-time recordings of user interactions.

Heatmaps: Visualize click, scroll, and attention data.

Form Analytics: Identify where users abandon forms.

Pricing: Free plan available; paid plans start at $24/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Mouseflow’s insights have been instrumental in improving our site’s usability.” – G2 Reviewer

17. Kissmetrics User Tracking Tool

Best for: Ecommerce businesses focused on customer behavior analytics.

What It Does: Kissmetrics tracks individual and group behaviors, offering insights to boost conversions and retention.

Standout Features:

Behavioral Analytics: Understand user actions and trends.

Funnel Reporting: Analyze conversion paths.

Cohort Analysis: Monitor user engagement over time.

Pricing: Custom pricing; contact Kissmetrics for details.

Rating: ★★★★☆ (4.3/5)

Reviews:

“Kissmetrics has provided us with a deeper understanding of our customer journey.” – Trustpilot Reviewer

18. Aloware Visitor Tracking & Communication Tool

Best for: Businesses focused on lead engagement through multi-channel communication.

What It Does: Aloware combines visitor tracking with communication tools like SMS, calls, and email to engage leads in real-time. It’s designed for businesses that prioritize immediate responses to high-intent visitors.

Standout Features:

Call Tracking: See how phone interactions connect to website activity.

Automated Follow-Ups: Respond to visitor inquiries instantly with SMS or email.

CRM Integration: Syncs with platforms like HubSpot, Salesforce, and more.

Pricing: Custom pricing based on business needs.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Aloware’s ability to connect visitor activity with communication tools is a game-changer for our team.” – G2 Reviewer

19. Albacross Company Visitor Tracking Tool

Best for: B2B marketers aiming to convert anonymous visitors into leads.

What It Does: Albacross identifies companies visiting your site and provides contact information for key decision-makers.

Standout Features:

Visitor Identification: Uncover anonymous website visitors.

Lead Generation: Access contact details of potential clients.

Analytics Dashboard: Monitor visitor behavior and engagement.

Pricing: Free plan available; paid plans start at €79/month.

Rating: ★★★★☆ (4.3/5)

Reviews:

“Albacross has significantly increased our lead conversion rates.” – G2 Reviewer

20. HubSpot Analytics Tool

Best for: Businesses seeking an all-in-one marketing and analytics platform.

What It Does: HubSpot Analytics offers comprehensive tracking of website performance, user behavior, and marketing campaigns.

Standout Features:

Traffic Analytics: Monitor website visits and sources.

Behavioral Events: Track specific user actions.

Custom Reporting: Create tailored reports to measure success.

Pricing: Free tools available; premium plans start at $50/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“HubSpot’s analytics have provided us with invaluable insights into our marketing efforts.” – Trustpilot Reviewer

21. Salesforce Marketing Cloud Interaction Studio

Best for: Enterprises aiming for personalized customer experiences.

What It Does: Interaction Studio tracks real-time customer interactions across channels to deliver personalized content.

Standout Features:

Real-Time Interaction Management: Engage customers at optimal moments.

Personalization: Tailor content based on user behavior.

Journey Mapping: Visualize and optimize customer journeys.

Pricing: Custom pricing; contact Salesforce for details.

Rating: ★★★★☆ (4.4/5)

Reviews:

“Interaction Studio has elevated our customer engagement strategies.” – G2 Reviewer

22. Zoho PageSense Heatmap & Visitor Tracking

Best for: Small to medium-sized businesses focusing on conversion rate optimization.

What It Does: Zoho PageSense offers tools like heatmaps, A/B testing, and funnel analysis to enhance website performance.

Standout Features:

Heatmaps: Visualize user engagement.

A/B Testing: Experiment with different page versions.

Goal Tracking: Measure conversions and key metrics.

Pricing: Plans start at $16/month.

Rating: ★★★★☆ (4.3/5)

Reviews:

“PageSense has been instrumental in optimizing our landing pages.” – Capterra Reviewer

23. Segment CDP & Visitor Tracking Tool

Best for: Businesses needing a robust customer data platform.

What It Does: Segment collects and unifies customer data from various sources, enabling better analytics and personalization.

Standout Features:

Data Collection: Gather data from websites, apps, and more.

Integrations: Connect with numerous analytics and marketing tools.

User Profiles: Build comprehensive customer profiles.

Pricing: Free plan available; paid plans start at $120/month.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Segment has streamlined our data collection and integration processes.” – G2 Reviewer

24. VWO (Visual Website Optimizer) Testing & Tracking Tool

Best for: Businesses focused on continuous website testing and optimization.

What It Does: VWO offers A/B testing, heatmaps, and analytics to help businesses improve conversion rates.

Standout Features:

A/B Testing: Test multiple variations to find the most effective.

Heatmaps and Clickmaps: Identify engagement trends.

Personalization Features: Deliver custom content to specific audience segments.

Pricing: Custom pricing; contact VWO for details.

Rating: ★★★★☆ (4.4/5)

Reviews:

“VWO has helped us refine our site’s design and boost conversions significantly.” – Capterra Reviewer

25. Chartbeat Analytics & Tracking Tool

Best for: Content-driven businesses tracking engagement in real time.

What It Does: Chartbeat monitors visitor engagement and behavior, helping publishers optimize content strategy.

Standout Features:

Real-Time Analytics: See what’s resonating with your audience as it happens.

Content Optimization: Tools to refine headlines, layouts, and more.

Audience Insights: Understand reader engagement and loyalty.

Pricing: Custom pricing based on business needs.

Rating: ★★★★☆ (4.5/5)

Reviews:

“Chartbeat has been instrumental in driving our content decisions.” – G2 Reviewer

What’s Next for Visitor Tracking Tools?

Visitor tracking tools have already come a long way, but they’re far from done evolving. As ecommerce and digital marketing continue to push boundaries, tracking tools are keeping pace with innovative features and advancements. 

Here’s what’s on the horizon:

1. Smarter Insights with Machine Learning

Machine learning is already making waves but the next generation of tracking tools will take it even further. Imagine tools that not only analyze user behavior but predict it – forecasting bounce risks, suggesting personalized offers, or optimizing entire customer journeys in real time.

Example: Machine learning could flag customers showing signs of churn and automatically trigger a retention campaign, saving revenue before it’s lost.

2. Seamless Cross-Device Tracking

Today’s shoppers bounce between devices like it’s a sport and tracking tools need to keep up. Expect advancements in cross-device tracking that deliver a more cohesive understanding of user journeys – whether someone starts on mobile, browses on a tablet, and finally checks out on a laptop.

Impact: With improved cross-device capabilities, ecommerce brands can deliver consistent, personalized experiences no matter where users engage.

3. A Bigger Focus on User Consent and Transparency

With privacy regulations like GDPR and CCPA becoming the norm, the future of visitor tracking is all about balancing insights with integrity. Tools will likely incorporate more transparent consent mechanisms, empowering users to manage how their data is collected and used.

The Future: Expect to see interactive dashboards where users can view, adjust, or even delete their data—a shift that builds trust while staying compliant.

The future of visitor tracking isn’t just about better data. It’s about smarter, more ethical ways of understanding and engaging users. 

For ecommerce marketers, these advancements mean tools that deliver more actionable insights while staying ahead of changing expectations and regulations. 

The future of visitor tracking is all about giving you smarter tools to understand your audience and make decisions that drive real results…without sacrificing trust or transparency.

Interested in learning more about Customers.ai visitor tracking tools can help your business? Try it free and start tracking visitors in 90 seconds!

See Who Is On Your Site Right Now!

Get names, emails, phone numbers & more.

Try it Free, No Credit Card Required

Start Your Free Trial

Important Next Steps

See what targeted outbound marketing is all about. Capture and engage your first 500 website visitor leads with Customers.ai X-Ray website visitor identification for free.

Talk and learn about sales outreach automation with other growth enthusiasts. Join Customers.ai Island, our Facebook group of 40K marketers and entrepreneurs who are ready to support you.

Advance your marketing performance with Sales Outreach School, a free tutorial and training area for sales pros and marketers.

The post Visitor Tracking Tools: The 25 Best Solutions for 2025 [Reviewed] appeared first on Customers.ai.

aiOla Releases Whisper-NER: An Open Source AI Model for Joint Speech T …

Speech recognition technology has made significant progress, with advancements in AI improving accessibility and accuracy. However, it still faces challenges, particularly in understanding spoken entities like names, places, and specific terminology. The issue is not only about converting speech to text accurately but also about extracting meaningful context in real-time. Current systems often require separate tools for transcription and entity recognition, leading to delays, inefficiencies, and inconsistencies. Additionally, privacy concerns regarding the handling of sensitive information during speech transcription present significant challenges for industries dealing with confidential data.

aiOla has released Whisper-NER: an open-source AI model that allows joint speech transcription and entity recognition. This model combines speech-to-text transcription with Named Entity Recognition (NER) to deliver a solution that can recognize important entities while transcribing spoken content. This integration allows for a more immediate understanding of context, making it suitable for industries requiring accurate and privacy-conscious transcription services, such as healthcare, customer service, and legal domains. Whisper-NER effectively combines transcription accuracy with the ability to identify and manage sensitive information.

Technical Details

Whisper-NER is based on the Whisper architecture developed by OpenAI, which is enhanced to perform real-time entity recognition while transcribing. By leveraging transformers, Whisper-NER can recognize entities like names, dates, locations, and specialized terminology directly from the audio input. The model is designed to work in real-time, which is valuable for applications that need instant transcription and comprehension, such as live customer support. Additionally, Whisper-NER incorporates privacy measures to obscure sensitive data, thereby enhancing user trust. The open-source nature of Whisper-NER also makes it accessible to developers and researchers, encouraging further innovation and customization.

The importance of Whisper-NER lies in its capability to deliver both accuracy and privacy. In tests, the model has shown a reduction in error rates compared to separate transcription and entity recognition models. According to aiOla, Whisper-NER provides a nearly 20% improvement in entity recognition accuracy and offers automatic redaction capabilities for sensitive data in real-time. This feature is particularly relevant for sectors like healthcare, where patient privacy must be protected, or for business settings, where confidential client information is discussed. The combination of transcription and entity recognition reduces the need for multiple steps in the workflow, providing a more streamlined and efficient process. It addresses a gap in speech recognition by enabling real-time comprehension without compromising security.

Conclusion

aiOla’s Whisper-NER represents an important step forward for speech recognition technology. By integrating transcription and entity recognition into one model, aiOla addresses the inefficiencies of current systems and provides a practical solution to privacy concerns. Its open-source availability means that the model is not only a tool but also a platform for future innovation, allowing others to build upon its capabilities. Whisper-NER’s contributions to enhancing transcription accuracy, protecting sensitive data, and improving workflow efficiencies make it a notable advancement in AI-powered speech solutions. For industries seeking an effective, accurate, and privacy-conscious solution, Whisper-NER sets a solid standard.

Check out the Paper, Model on Hugging Face, and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post aiOla Releases Whisper-NER: An Open Source AI Model for Joint Speech Transcription and Entity Recognition appeared first on MarkTechPost.

CMU Researchers Propose XGrammar: An Open-Source Library for Efficient …

The field of structured generation has become important with the rise of LLMs. These models, capable of generating human-like text, are now tasked with producing outputs that follow rigid formats such as JSON, SQL, and other domain-specific languages. Applications like code generation, robotic control, and structured querying depend heavily on these capabilities. However, ensuring that outputs conform to specific structures without compromising speed or efficiency remains a significant challenge. Structured outputs allow for seamless downstream processing, but the complexity of achieving these results necessitates innovative solutions.

Despite advancements in LLMs, structured output generation continues to be plagued by inefficiencies. One major challenge is managing the computational demands of adhering to grammatical constraints during output generation. Traditional methods like context-free grammar (CFG) interpretation require processing each possible token in the model’s vocabulary, which can exceed 128,000 tokens. Moreover, maintaining stack states to track recursive grammar rules adds to runtime delays. As a result, existing systems often experience high latency and increased resource usage, making them unsuitable for real-time or large-scale applications.

Current tools for structured generation utilize constrained decoding methods to ensure outputs align with predefined rules. These approaches filter out invalid tokens by setting their probabilities to zero at each decoding step. While effective, constrained decoding often needs to improve its efficiency due to evaluating each token against the entire stack state. Also, the recursive nature of CFGs further complicates runtime processing. These challenges have limited the scalability and practicality of existing systems, particularly when handling complex structures or large vocabularies.

Researchers from Carnegie Mellon University, NVIDIA, Shanghai Jiao Tong University, and the University of California Berkeley developed XGrammar, a groundbreaking structured generation engine to address these limitations. XGrammar introduces a novel approach by dividing tokens into two categories: context-independent tokens that can be prevalidated and context-dependent tokens requiring runtime evaluation. This separation significantly reduces the computational burden during output generation. Also, the system incorporates a co-designed grammar and inference engine, enabling it to overlap grammar computations with GPU-based LLM operations, thereby minimizing overhead.

XGrammar’s technical implementation includes several key innovations. It uses a byte-level pushdown automaton to process CFGs efficiently, enabling it to handle irregular token boundaries and nested structures. The adaptive token mask cache precomputes and stores validity for context-independent tokens, covering over 99% of tokens in most cases. Context-dependent tokens, representing less than 1% of the total, are processed using a persistent execution stack that allows for rapid branching and rollback operations. XGrammar’s preprocessing phase overlaps with the LLM’s initial prompt processing, ensuring near-zero latency for structured generation.

Image Source

Performance evaluations reveal the significant advantages of XGrammar. For JSON grammar tasks, the system achieves a token mask generation time of less than 40 microseconds, delivering up to a 100x speedup compared to traditional methods. Integrated with the Llama 3.1 model, XGrammar enables an 80x improvement in end-to-end structured output generation on the NVIDIA H100 GPU. Moreover, memory optimization techniques reduce storage requirements to just 0.2% of the original size, from 160 MB to 0.46 MB. These results demonstrate XGrammar’s ability to handle large-scale tasks with unprecedented efficiency.

The researchers’ efforts have several key takeaways:

Token Categorization: By precomputing context-independent tokens and reducing runtime checks for context-dependent tokens, XGrammar significantly minimizes computational overhead.

Memory Efficiency: The adaptive token mask cache reduces memory usage to just 0.2% of the original requirements, making it highly scalable.

Enhanced Performance: With a 100x speedup in CFG processing and an 80x improvement in structured output generation, XGrammar sets a new benchmark for efficiency.

Cross-Platform Deployment: XGrammar supports a wide range of platforms, including client-side browsers, enabling its use in portable devices like smartphones.

Integration with LLM Frameworks: The system seamlessly integrates with popular LLM models, such as Llama 3.1, ensuring compatibility and ease of adoption.

Image Source

In conclusion, XGrammar represents a transformative step in structured generation for large language models. Addressing inefficiencies in traditional CFG processing and constrained decoding offers a scalable, high-performance solution for generating structured outputs. Its innovative techniques, such as token categorization, memory optimization, and platform compatibility, make it an essential tool for advancing AI applications. With results up to 100x speedup and reduced latency, XGrammar sets a new standard for structured generation, enabling LLMs to meet modern AI systems’ demands effectively.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post CMU Researchers Propose XGrammar: An Open-Source Library for Efficient, Flexible, and Portable Structured Generation appeared first on MarkTechPost.

Meet CircleMind: An AI Startup that is Transforming Retrieval Augmente …

In an era of information overload, advancing AI requires not just innovative technologies but smarter approaches to data processing and understanding. Meet CircleMind, an AI startup reimagining Retrieval Augmented Generation (RAG) by using knowledge graphs and the established PageRank algorithm. Funded by Y Combinator, CircleMind aims to improve how large language models (LLMs) understand and generate content by providing a more structured and nuanced approach to information retrieval. Let’s take a closer look at how this works and why it matters.

For those unfamiliar with RAG, it’s an AI technique that blends information retrieval with language generation. Typically, a large language model like GPT-3 will respond to queries based on its training data, which, though vast, is inevitably outdated or incomplete over time. RAG augments this by pulling in real-time or domain-specific data during the generation process—essentially a smart mix of search engine functionality with conversational fluency.

Traditional RAG models often rely on keyword-based searches or dense vector embeddings, which may lack contextual sophistication. This can lead to a flood of data points without ensuring that the most relevant, authoritative sources are prioritized, resulting in responses that may not be reliable. CircleMind aims to solve this problem by introducing more sophisticated information retrieval techniques.

The CircleMind Approach: Knowledge Graphs and PageRank

CircleMind’s approach revolves around two key technologies: Knowledge Graphs and the PageRank Algorithm.

Knowledge graphs are structured networks of interconnected entities—think people, places, organizations—designed to represent the relationships between various concepts. They help machines not just identify words but understand their connections, thereby elevating how context is both interpreted and applied during the generation of responses. This richer representation of relationships helps CircleMind retrieve data that is more nuanced and contextually accurate.

However, understanding relationships is only part of the solution. CircleMind also leverages the PageRank algorithm, a technique developed by Google’s founders in the late 1990s that measures the importance of nodes within a graph based on the quantity and quality of incoming links. Applied to a knowledge graph, PageRank can prioritize nodes that are more authoritative and well-connected. In CircleMind’s context, this ensures that the retrieved information is not only relevant but also carries a measure of authority and trustworthiness.

By combining these two techniques, CircleMind enhances both the quality and reliability of the information retrieved, providing more contextually appropriate data for LLMs to generate responses.

The Advantage: Relevance, Authority, and Precision

By combining knowledge graphs and PageRank, CircleMind addresses some key limitations of conventional RAG implementations. Traditional models often struggle with context ambiguity, while knowledge graphs help CircleMind represent relationships more richly, leading to more meaningful and accurate responses.

PageRank, meanwhile, helps prioritize the most important information from a graph, ensuring that the AI’s responses are both relevant and dependable. By combining these approaches, CircleMind’s RAG ensures that the AI retrieves contextually relevant and reliable data, leading to informative and accurate responses. This combination significantly enhances the ability of AI systems to understand not only what information is relevant, but also which sources are authoritative.

Practical Implications and Use Cases

The benefits of CircleMind’s approach become most apparent in practical use cases where precision and authority are critical. Enterprises seeking AI for customer service, research assistance, or internal knowledge management will find CircleMind’s methodology valuable. By ensuring that an AI system retrieves authoritative, contextually nuanced information, the risk of incorrect or misleading responses is reduced—a critical factor for applications like healthcare, financial advisory, or technical support, where accuracy is essential.

CircleMind’s architecture also provides a strong framework for domain-specific AI solutions, particularly those that require nuanced understanding across large sets of interrelated data. For instance, in the legal field, an AI assistant could use CircleMind’s approach to not only pull in relevant case law but also understand the precedents and weigh their authority based on real-world legal outcomes and citations. This ensures that the information presented is both accurate and contextually applicable, making the AI’s output more trustworthy.

A Nod to the Old and New

CircleMind’s innovation is as much a nod to the past as it is to the future. By reviving and repurposing PageRank, CircleMind demonstrates that significant advancements often come from iterating and integrating existing technologies in innovative ways. The original PageRank created a hierarchy of web pages based on interconnectedness; CircleMind similarly creates a more meaningful hierarchy of information, tailored for generative models.

The use of knowledge graphs acknowledges that the future of AI is about smarter models that understand how data is interconnected. Rather than relying solely on bigger models with more data, CircleMind focuses on relationships and context, providing a more sophisticated approach to information retrieval that ultimately leads to more intelligent response generation.

The Road Ahead

CircleMind is still in its early stages, and realizing the full potential of its technology will take time. The main challenge lies in scaling this hybrid RAG approach without sacrificing speed or incurring prohibitive computational costs. Dynamic integration of knowledge graphs in real-time queries and ensuring efficient computation or approximation of PageRank will require both innovative engineering and significant computational resources.

Despite these challenges, the potential for CircleMind’s approach is clear. By refining RAG, CircleMind aims to bridge the gap between raw data retrieval and nuanced content generation, ensuring that retrieved content is contextually rich, accurate, and authoritative. This is particularly crucial in an era where misinformation and lack of reliability are persistent issues for generative models.

The future of AI is not merely about retrieving information, but about understanding its context and significance. CircleMind is making meaningful progress in this direction, offering a new paradigm for information retrieval in language generation. By integrating knowledge graphs and leveraging the established strengths of PageRank, CircleMind is paving the way for AI to deliver not only answers but informed, trustworthy, and context-aware guidance.

Check out the details here. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Meet CircleMind: An AI Startup that is Transforming Retrieval Augmented Generation with Knowledge Graphs and PageRank appeared first on MarkTechPost.

Researchers from the University of Maryland and Adobe Introduce DynaSa …

Traditional large language model (LLM) agent systems face significant challenges when deployed in real-world scenarios due to their limited flexibility and adaptability. Existing LLM agents typically select actions from a predefined set of possibilities at each decision point, a strategy that works well in closed environments with narrowly scoped tasks but falls short in more complex and dynamic settings. This static approach not only restricts the agent’s capabilities but also requires considerable human effort to anticipate and implement every potential action beforehand, which becomes impractical for complex or evolving environments. Consequently, these agents are unable to adapt effectively to new, unforeseen tasks or solve long-horizon problems, highlighting the need for more robust, self-evolving capabilities in LLM agents.

Researchers from the University of Maryland and Adobe introduce DynaSaur: an LLM agent framework that enables the dynamic creation and composition of actions online. Unlike traditional systems that rely on a fixed set of predefined actions, DynaSaur allows agents to generate, execute, and refine new Python functions in real-time whenever existing functions prove insufficient. The agent maintains a growing library of reusable functions, enhancing its ability to respond to diverse scenarios. This dynamic ability to create, execute, and store new tools makes AI agents more adaptable to real-world challenges.

Technical Details

The technical backbone of DynaSaur revolves around the use of Python functions as representations of actions. Each action is modeled as a Python snippet, which the agent generates, executes, and assesses in its environment. If existing functions do not suffice, the agent dynamically creates new ones and adds them to its library for future reuse. This system leverages Python’s generality and composability, allowing for a flexible approach to action representation. Furthermore, a retrieval mechanism allows the agent to fetch relevant actions from its accumulated library using embedding-based similarity search, addressing context length limitations and improving efficiency.

DynaSaur also benefits from integration with the Python ecosystem, giving the agent the ability to interact with a variety of tools and systems. Whether it needs to access web data, manipulate file contents, or execute computational tasks, the agent can write or reuse functions to fulfill these demands without human intervention, demonstrating a high level of adaptability.

The significance of DynaSaur lies in its ability to overcome the limitations of predefined action sets and thereby enhance the flexibility of LLM agents. In experiments on the GAIA benchmark, which evaluates the adaptability and generality of AI agents across a broad spectrum of tasks, DynaSaur outperformed all baselines. Using GPT-4, it achieved an average accuracy of 38.21%, surpassing existing methods. When combining human-designed tools with its generated actions, DynaSaur showed an 81.59% improvement, highlighting the synergy between expert-crafted tools and dynamically generated ones.

Notably, strong performance was observed in complex tasks categorized under Level 2 and Level 3 of the GAIA benchmark, where DynaSaur’s ability to create new actions allowed it to adapt and solve problems beyond the scope of predefined action libraries. By achieving the top position on the GAIA public leaderboard, DynaSaur has set a new standard for LLM agents in terms of adaptability and efficiency in handling unforeseen challenges.

Conclusion

DynaSaur represents a significant advancement in the field of LLM agent systems, offering a new approach where agents are not just passive entities following predefined scripts but active creators of their own tools and capabilities. By dynamically generating Python functions and building a library of reusable actions, DynaSaur enhances the adaptability, flexibility, and problem-solving capacity of LLMs, making them more effective for real-world tasks. This approach addresses the limitations of current LLM agent systems and opens new avenues for developing AI agents that can autonomously evolve and improve over time. DynaSaur thus paves the way for more practical, robust, and versatile AI applications across a wide range of domains.

Check out the Paper and GitHub Page. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Researchers from the University of Maryland and Adobe Introduce DynaSaur: The LLM Agent that Grows Smarter by Writing its Own Functions appeared first on MarkTechPost.

LTX-Video: A Groundbreaking Real-Time Video Generation Open-Source Mod …

Lightricks, a company renowned for its innovative technological advancements in creative tools, has unveiled its groundbreaking video generation open-source model, LTX Video (LTXV). Setting a benchmark for the industry, LTXV was released with native support in ComfyUI on the very first day. This significant launch has sparked excitement in the tech and creative communities, promising to redefine real-time video generation capabilities.

LTXV, a cutting-edge video generation model, is based on a DiT (Diffusion Transformer) architecture and boasts an impressive 2 billion parameters. Its unparalleled ability to generate high-quality videos in real-time marks a turning point in creative technology. Operating at 24 frames per second (FPS) with a resolution of 768×512 pixels, LTXV generates video content faster than it can be viewed. This efficiency offers revolutionary prospects for creators seeking speed and precision in their projects.

Image Source

Features That Redefine Standards

Real-time Generation Speed: LTXV is engineered for speed at its core. It can produce five seconds of video (121 frames) in only four seconds, demonstrating a remarkable advantage in real-time applications. This capability ensures content creators can seamlessly generate videos without significant processing delays, catering to time-sensitive workflows. LTXV achieves this without compromising on visual fidelity or memory efficiency. The model is optimized for widely available hardware, such as the NVIDIA RTX 4090, and utilizes bfloat16 precision. This ensures minimal memory usage while maintaining the highest quality standards.

Unmatched Video Quality: LTXV takes video quality to new heights. Its Diffusion Transformer architecture guarantees smooth motion and eliminates issues like object morphing, ensuring exceptional motion consistency. The model’s scalability allows it to generate extended videos while maintaining uniform quality, enabling creators to tell compelling long-form stories without technical constraints.

Seamless Integration with ComfyUI: Underscoring its user-centric approach, Lightricks has ensured LTXV’s native integration with ComfyUI. This includes dedicated nodes branded as “LTXVideo,” conveniently available within the ComfyUI Manager. Users can search for “LTXVideo” to access these custom nodes, streamlining the workflow for immediate creative exploration.

Integrating LTXV with ComfyUI is a technical milestone and a catalyst for creative innovation. By enabling high-fidelity real-time video generation, LTXV opens doors to new forms of storytelling and artistic expression. Its seamless functionality empowers artists, filmmakers, and designers to experiment with video content creation at an unprecedented scale. Lightricks’ commitment to accessibility and innovation is evident in its focus on optimizing LTXV for commonly available GPUs.

Image Source

The ComfyUI community, known for its dynamic engagement and creative ingenuity, is poised to explore the full spectrum of possibilities LTXV offers. By combining LTXV’s real-time capabilities with ComfyUI’s robust interface, creators can experiment and push the boundaries of what is achievable in video generation. LTXV’s release also underscores a broader trend in creative technology: democratizing advanced tools. By making state-of-the-art models accessible to a wider audience, companies like Lightricks foster an environment where innovation thrives across disciplines.

In conclusion, the launch of LTX Video (LTXV) and its native integration with ComfyUI from day one marks a pivotal moment in the evolution of creative tools. With its real-time speed, unmatched video quality, and seamless integration, LTXV sets a new standard for video generation models. Its user-centric approach, combined with the support for widely available hardware, ensures that creators from all backgrounds can harness its potential. The creative possibilities are boundless as the ComfyUI community begins to explore and innovate with LTXV. Lightricks has introduced a powerful tool and paved the way for a new storytelling and content creation era.

Check out the Paper and Model. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post LTX-Video: A Groundbreaking Real-Time Video Generation Open-Source Model with Day-One Native Support in ComfyUI, Empowering Innovators to Transform Content Creation appeared first on MarkTechPost.

Researchers from MBZUAI and CMU Introduce Bi-Mamba: A Scalable and Eff …

The evolution of machine learning has brought significant advancements in language models, which are foundational to tasks like text generation and question-answering. Among these, transformers and state-space models (SSMs) are pivotal, yet their efficiency when handling long sequences has posed challenges. As sequence length increases, traditional transformers suffer from quadratic complexity, leading to prohibitive memory and computational demands. To address these issues, researchers and organizations have explored alternative architectures, such as Mamba, a state-space model with linear complexity that provides scalability and efficiency for long-context tasks.

Large-scale language models often face challenges in managing computational costs, especially as they scale up to billions of parameters. For instance, while Mamba offers linear complexity advantages, its increasing size results in significant energy consumption and training costs, making deployment difficult. These limitations are exacerbated by the high resource demands of models like GPT-based architectures, which are traditionally trained and inferred at full precision (e.g., FP16 or BF16). Moreover, as demand grows for efficient, scalable AI, exploring extreme quantization methods has become critical to ensure practical deployment in resource-constrained settings.

Researchers have explored techniques such as pruning, low-bit quantization, and key-value cache optimizations to mitigate these challenges. Quantization, which reduces the bit-width of model weights, has shown promising results by compressing models without substantial performance degradation. However, most of these efforts focus on transformer-based models. The behavior of SSMs, particularly Mamba, under extreme quantization still needs to be explored, creating a gap in developing scalable and efficient state-space models for real-world applications.

Researchers from the Mohamed bin Zayed University of Artificial Intelligence and Carnegie Mellon University introduced Bi-Mamba, a 1-bit scalable Mamba architecture designed for low-memory, high-efficiency scenarios. This innovative approach applies binarization-aware training to Mamba’s state-space framework, enabling extreme quantization while maintaining competitive performance. Bi-Mamba was developed in model sizes of 780 million, 1.3 billion, and 2.7 billion parameters and trained from scratch using an autoregressive distillation loss. The model uses high-precision teacher models such as LLaMA2-7B to guide training, ensuring robust performance.

The architecture of Bi-Mamba employs selective binarization of its linear modules while retaining other components at full precision to balance efficiency and performance. Input and output projections are binarized using FBI-Linear modules, which integrate learnable scaling and shifting factors for optimal weight representation. This ensures that binarized parameters align closely with their full-precision counterparts. The model’s training utilized 32 NVIDIA A100 GPUs to process large datasets, including 1.26 trillion tokens from sources like RefinedWeb and StarCoder.

Extensive experiments demonstrated Bi-Mamba’s competitive edge over existing models. On datasets like Wiki2, PTB, and C4, Bi-Mamba achieved perplexity scores of 14.2, 34.4, and 15.0, significantly outperforming alternatives like GPTQ and Bi-LLM, which exhibited perplexities up to 10× higher. Also, Bi-Mamba achieved zero-shot accuracies of 44.5% for the 780M model, 49.3% for the 2.7B model, and 46.7% for the 1.3B variant on downstream tasks such as BoolQ and HellaSwag. This demonstrated its robustness across various tasks and datasets while maintaining energy-efficient performance.

The study’s findings highlight several key takeaways:

Efficiency Gains: Bi-Mamba achieves over 80% storage compression compared to full-precision models, reducing storage size from 5.03GB to 0.55GB for the 2.7B model.

Performance Consistency: The model retains comparable performance to full-precision counterparts with significantly reduced memory requirements.

Scalability: Bi-Mamba’s architecture enables effective training across multiple model sizes, with competitive results even for the largest variants.

Robustness in Binarization: By selectively binarizing linear modules, Bi-Mamba avoids the performance degradation typically associated with naive binarization methods.

In conclusion, Bi-Mamba represents a significant step forward in addressing the dual challenges of scalability and efficiency in large language models. By leveraging binarization-aware training and focusing on key architectural optimizations, the researchers demonstrated that state-space models could achieve high performance under extreme quantization. This innovation enhances energy efficiency, reduces resource consumption, and sets the stage for future developments in low-bit AI systems, opening avenues for deploying large-scale models in practical, resource-limited environments. Bi-Mamba’s robust results underscore its potential as a transformative approach for more sustainable and efficient AI technologies.

Check out the Paper. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Researchers from MBZUAI and CMU Introduce Bi-Mamba: A Scalable and Efficient 1-bit Mamba Architecture Designed for Large Language Models in Multiple Sizes (780M, 1.3B, and 2.7B Parameters) appeared first on MarkTechPost.

Google Upgrades Gemini-exp-1121: Advancing AI Performance in Coding, M …

The field of artificial intelligence (AI) continues to evolve, with competition among large language models (LLMs) remaining intense. Despite recent advances pushing the boundaries of what these models can achieve, challenges persist. One of the main difficulties for existing LLMs, such as GPT-4, is finding the right balance between general-purpose reasoning, coding abilities, and visual understanding. Many models excel in one domain while underperforming in others, making it challenging for developers and researchers to find a single model that can effectively address diverse needs. This creates inefficiencies and highlights the need for more versatile solutions.

Gemini-exp-1121: A Notable Upgrade

Google has upgraded Gemini-exp-1121, which outperforms GPT-4o in coding, math, and vision by 20%. Gemini-exp-1121 is the latest experimental addition to Google’s Gemini series of AI models, designed to meet the growing demand for a comprehensive AI system. Compared to OpenAI’s GPT-4o, Gemini-exp-1121 has shown notable improvements, particularly in coding, mathematical reasoning, and visual understanding. This upgrade represents a substantial advancement, enhancing Google’s standing in the AI ecosystem alongside OpenAI. Gemini-exp-1121 aims to address gaps in previous LLM capabilities by improving coding fluency, enhancing complex problem-solving abilities, and refining perceptual skills.

Image taken on Nov 22 2024: Source https://lmarena.ai/

Technical Improvements and Benefits

Technically, Gemini-exp-1121 includes several significant improvements. These enhancements involve optimized transformer architecture and advanced retrieval mechanisms to augment its learning with real-time data, helping the model remain current and accurate. The improvement in coding performance is attributed to extensive fine-tuning using real-world programming data from various languages and frameworks. Additionally, the model benefits from enhanced algorithms for reasoning capabilities, using deeper context analysis to solve complex math problems more effectively. Its improved visual understanding is facilitated by a multimodal architecture capable of processing both text and image inputs seamlessly, making it suitable for tasks like visual storytelling and generating code based on design sketches.

The impact of Gemini-exp-1121 goes beyond technical improvements; it influences how developers and data scientists approach problem-solving. Google’s experiments indicate that Gemini-exp-1121 performs coding tasks with a higher success rate compared to GPT-4o, achieving around a 20% increase in correct outputs on benchmark problems. Its visual understanding capabilities also enable it to generate descriptions and contextual inferences with greater precision than its predecessors. These advances make it a useful tool for enterprises looking to automate workflows involving both code and visual components, such as app development and product design. The focus on enhanced reasoning capabilities also makes Gemini-exp-1121 promising for educational and research settings where sophisticated problem-solving skills are essential.

Conclusion

Google’s Gemini-exp-1121 represents an important step forward in the LLM space by addressing performance gaps in multiple domains that have traditionally been challenging for AI models. Its 20% improvement in key areas such as coding, math, and vision offers practical benefits in various applications, making it a strong competitor to GPT-4o. By integrating enhanced reasoning, improved coding performance, and advanced visual processing, Google has positioned Gemini-exp-1121 as a versatile solution for many of the challenges faced by AI practitioners today. This progress highlights the ongoing development in AI capabilities, promising more efficient and versatile tools for professionals across industries.

Check out the Details here. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Google Upgrades Gemini-exp-1121: Advancing AI Performance in Coding, Math, and Visual Understanding appeared first on MarkTechPost.

Apple Releases AIMv2: A Family of State-of-the-Art Open-Set Vision Enc …

Vision models have evolved significantly over the years, with each innovation addressing the limitations of previous approaches. In the field of computer vision, researchers have often faced challenges in balancing complexity, generalizability, and scalability. Many current models struggle to effectively handle diverse visual tasks or adapt efficiently to new datasets. Traditionally, large-scale pre-trained vision encoders have used contrastive learning, which, despite its success, presents challenges in scaling and parameter efficiency. There remains a need for a robust, versatile model that can handle multiple modalities—such as images and text—without sacrificing performance or requiring extensive data filtering.

AIMv2: A New Approach

Apple has taken on this challenge with the release of AIMv2, a family of open-set vision encoders designed to improve upon existing models in multimodal understanding and object recognition tasks. Inspired by models like CLIP, AIMv2 adds an autoregressive decoder, allowing it to generate image patches and text tokens. The AIMv2 family includes 19 models with varying parameter sizes—300M, 600M, 1.2B, and 2.7B—and supports resolutions of 224, 336, and 448 pixels. This range in model size and resolution makes AIMv2 suitable for different use cases, from smaller-scale applications to tasks requiring larger models.

Technical Overview

AIMv2 incorporates a multimodal autoregressive pre-training framework, which builds on the conventional contrastive learning approach used in similar models. The key feature of AIMv2 is its combination of a Vision Transformer (ViT) encoder with a causal multimodal decoder. During pre-training, the encoder processes image patches, which are subsequently paired with corresponding text embeddings. The causal decoder then autoregressively generates both image patches and text tokens, reconstructing the original multimodal inputs. This setup simplifies training and facilitates model scaling without requiring specialized inter-batch communication or extremely large batch sizes. Additionally, the multimodal objective allows AIMv2 to achieve denser supervision compared to other methods, enhancing its ability to learn from both image and text inputs.

Performance and Scalability

AIMv2 outperforms major existing models like OAI CLIP and SigLIP on most multimodal understanding benchmarks. Specifically, AIMv2-3B achieved 89.5% top-1 accuracy on the ImageNet dataset with a frozen trunk, demonstrating notable robustness in frozen encoder models. Compared to DINOv2, AIMv2 also performed well in open-vocabulary object detection and referring expression comprehension. Moreover, AIMv2’s scalability was evident, as its performance consistently improved with increasing data and model size. The model’s flexibility and integration with modern tools, such as the Hugging Face Transformers library, make it practical and straightforward to implement across various applications.

Conclusion

AIMv2 represents a meaningful advancement in the development of vision encoders, emphasizing simplicity in training, effective scaling, and versatility in multimodal tasks. Apple’s release of AIMv2 offers improvements over previous models, with strong performance on numerous benchmarks, including open-vocabulary recognition and multimodal tasks. The integration of autoregressive techniques enables dense supervision, resulting in robust and flexible model capabilities. AIMv2’s availability on platforms like Hugging Face allows developers and researchers to experiment with advanced vision models more easily. AIMv2 sets a new standard for open-set visual encoders, capable of addressing the increasing complexity of real-world multimodal understanding.

Check out the Paper and AIMv2 family of the models on Hugging Face. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Apple Releases AIMv2: A Family of State-of-the-Art Open-Set Vision Encoders appeared first on MarkTechPost.

Jina AI Introduces Jina-CLIP v2: A 0.9B Multilingual Multimodal Embedd …

In an interconnected world, effective communication across multiple languages and mediums is increasingly important. Multimodal AI faces challenges in combining images and text for seamless retrieval and understanding across different languages. Existing models often perform well in English but struggle with other languages. Additionally, handling high-dimensional data for both text and images simultaneously has been computationally intensive, limiting applications for non-English speakers and scenarios requiring multilingual contexts.

Jina-CLIP v2: A 0.9B Multilingual Multimodal Embedding Model

Jina AI has introduced Jina-CLIP v2—a 0.9B multilingual multimodal embedding model that connects images with text in 89 languages. Jina-CLIP v2 supports a wide range of languages, addressing the limitations that have previously restricted access to advanced multimodal AI technologies. It handles images at a resolution of 512×512 and processes text with up to 8,000 tokens, providing an effective solution for linking images and multilingual text. Additionally, it offers Matryoshka representations that reduce embeddings to 64 dimensions for both text and images, ensuring more efficient embeddings while retaining essential contextual information.

Technical Details

Jina-CLIP v2 stands out for its flexibility and efficiency. It enables embedding generation not only at a large dimensional scale but also at smaller scales, with its Matryoshka representation feature reducing embeddings to 64 dimensions. This allows users to adjust the embedding process to meet specific requirements, whether for computationally intensive deep learning tasks or lightweight mobile applications. Furthermore, the model’s text encoder can operate independently as a dense retriever, matching the performance of jina-embeddings-v3—the current leader for multilingual embeddings under 1 billion parameters on the Multilingual Text Embeddings Benchmark (MTEB). The versatility to perform both retrieval and classification tasks makes Jina-CLIP v2 suitable for a variety of use cases, from multilingual search engines to context-aware recommendation systems.

Jina-CLIP v2 represents an important step in reducing biases in language models, particularly for users relying on less widely spoken languages. In evaluations, the model performed well in multilingual retrieval tasks, demonstrating its capability to match or exceed the performance of specialized text models. Its use of Matryoshka representations ensures that embedding calculations can be performed efficiently without sacrificing accuracy, enabling deployment in resource-constrained environments. Jina-CLIP v2’s ability to connect text and images across 89 languages opens new possibilities for companies and developers to create AI that is accessible to diverse users while maintaining contextual accuracy. This can significantly impact applications in e-commerce, content recommendation, and visual search systems, where language barriers have traditionally posed challenges.

Conclusion

Jina-CLIP v2 is a meaningful advancement in multilingual multimodal models, addressing both linguistic diversity and technical efficiency in a unified approach. By enabling effective image and text connectivity across 89 languages, Jina AI is contributing to more inclusive AI tools that transcend linguistic boundaries. Whether for retrieval or classification tasks, Jina-CLIP v2 offers flexibility, scalability, and performance that empower developers to create robust and efficient AI applications. This development is a step forward in making AI accessible and effective for people around the world, fostering cross-cultural interactions and understanding.

Check out the details here. All credit for this research goes to the researchers of this project. Also, don’t forget to follow us on Twitter and join our Telegram Channel and LinkedIn Group. If you like our work, you will love our newsletter.. Don’t Forget to join our 55k+ ML SubReddit.

[FREE AI VIRTUAL CONFERENCE] SmallCon: Free Virtual GenAI Conference ft. Meta, Mistral, Salesforce, Harvey AI & more. Join us on Dec 11th for this free virtual event to learn what it takes to build big with small models from AI trailblazers like Meta, Mistral AI, Salesforce, Harvey AI, Upstage, Nubank, Nvidia, Hugging Face, and more.
The post Jina AI Introduces Jina-CLIP v2: A 0.9B Multilingual Multimodal Embedding Model that Connects Image with Text in 89 Languages appeared first on MarkTechPost.

Accelerating Mixtral MoE fine-tuning on Amazon SageMaker with QLoRA

Companies across various scales and industries are using large language models (LLMs) to develop generative AI applications that provide innovative experiences for customers and employees. However, building or fine-tuning these pre-trained LLMs on extensive datasets demands substantial computational resources and engineering effort. With the increase in sizes of these pre-trained LLMs, the model customization process becomes complex, time-consuming, and often prohibitively expensive for most organizations that lack the necessary infrastructure and skilled talent.
In this post, we demonstrate how you can address these challenges by using fully managed environment with Amazon SageMaker Training jobs to fine-tune the Mixtral 8x7B model using PyTorch Fully Sharded Data Parallel (FSDP) and Quantized Low Rank Adaptation (QLoRA).
We guide you through a step-by-step implementation of model fine-tuning on a GEM/viggo dataset, employing the QLoRA fine-tuning strategy on a single p4d.24xlarge worker node (providing 8 Nvidia A100 40GB GPUs).
Business challenge
Today’s businesses are looking to adopt a variety of LLMs to enhance business applications. Primarily, they’re looking for foundation models (FMs) that are open source (that is, model weights that work without modification from the start) and can offer computational efficiency and versatility. Mistral’s Mixtral 8x7B model, released with open weights under the Apache 2.0 license, is one of the models that has gained popularity with large enterprises due to the high performance that it offers across various tasks. Mixtral employs a sparse mixture of experts (SMoE) architecture, selectively activating only a subset of its parameters for each input during model training. This architecture allows these models to use only 13B (about 18.5%) of its 46.7B total parameters during inference, making it high performing and efficient.
These FMs work well for many use cases but lack domain-specific information that limits their performance at certain tasks. This requires businesses to use fine-tuning strategies to adapt these large FMs to specific domains, thus improving performance on targeted applications. Due to the growing number of model parameters and the increasing context lengths of these modern LLMs, this process is memory intensive and requires advanced AI expertise to align and optimize them effectively. The cost of provisioning and managing the infrastructure increases the overall cost of ownership of the end-to-end solution.
In the upcoming section, we discuss how you can cost-effectively build such a solution with advanced memory optimization techniques using Amazon SageMaker.
Solution overview
To address the memory challenges of fine-tuning LLMs such as Mixtral, we will adopt the QLoRA method. As shown in the following diagram, QLoRA freezes the original model’s weights and adds low-rank trainable parameters to the transformer layers. QLoRA further uses quantization to represent the actual model’s weights in a compact, optimized format such as 4-bit NormalFloat (NF4), effectively compressing the model and reducing its memory footprint. This enables training and fine-tuning these LLMs even on systems with limited memory while maintaining performance comparable to half-precision fine-tuning. QLoRA’s support for double quantization and paged optimizers reduces the memory footprint further by quantizing the quantization constants and effectively handling any sudden memory demands.
During the forward pass computation of this architecture, the 4-bit weights get dequantized to bfloat16 (BF16) precision. On the other hand, the LoRA adapters continue to operate on BF16 precision data. Both (original weights and adapter output vectors) are then added together element-wise to produce the final result, denoted as h.
During the backward pass of the model, the gradients are computed with respect to only the LoRA parameters, not the original base model weights. Although the dequantized original weights are used in calculations, the original 4-bit quantized weights of the base model remain unchanged.
To adopt the following architecture, we will use the Hugging Face Parameter-Efficent Fine-tuning (PEFT) library, which integrates directly with bitsandbytes. This way, the QLoRA technique to fine-tune can be adopted with just a few lines of code.
QLoRA operates on a large FM. In the figure below, X denotes the input tokens of the training data, W is the existing model weights (quantized), and Wa, Wb are the segments of the adapters added by QLoRA. The original model’s weights (W) are frozen, and QLoRA adds adapters (Wa, Wb), which are low-rank trainable parameters, onto the existing transformer layer.

Figure 1: This figure shows how QLoRA operates. The original model’s weights (W) are frozen, and QLoRA adds in adapters (Wa, Wb) onto the existing transformer layer.

Although QLoRA helps optimize memory during fine-tuning, we will use Amazon SageMaker Training to spin up a resilient training cluster, manage orchestration, and monitor the cluster for failures. By offloading the management and maintenance of the training cluster to SageMaker, we reduce both training time and our total cost of ownership (TCO). Using this approach, you can focus on developing and refining the model while using the fully managed training infrastructure provided by SageMaker Training.
Implementation details
We spin up the cluster by calling the SageMaker control plane through APIs or the AWS Command Line Interface (AWS CLI) or using the SageMaker AWS SDK. In response, SageMaker spins up training jobs with the requested number and type of compute instances. In our example, we use one ml.p4d.24xlarge compute instance.
To take complete advantage of this multi-GPU cluster, we use the recent support of QLoRA and PyTorch FSDP. Although QLoRA reduces computational requirements and memory footprint, FSDP, a data/model parallelism technique, will help shard the model across all eight GPUs (one ml.p4d.24xlarge), enabling training the model even more efficiently. Hugging Face PEFT is where the integration happens, and you can read more about it in the PEFT documentation.
QLoRA adapters are added to the linear layers in the model. The layers (for example, transformer layers, gate networks, and feed-forward networks) put together will form the entire model, as shown in the following diagram, which will be considered to be sharded by FSDP across our cluster (shown as small shards in blue).

The following architecture diagram shows how you can use SageMaker Training to have the SageMaker Control Plane spin up a resilient training job cluster. SageMaker downloads the training image from Amazon Elastic Container Registry (Amazon ECR) and will use Amazon Simple Storage Service (Amazon S3) as an input training data source and to store training artifacts.

Figure 3: Architecture Diagram showing how you can utilize SageMaker Training Jobs to spin up a resilient training cluster. Amazon ECR contains the training image, and Amazon S3 contains the training artifacts.

To put this solution into practice, execute the following use case.
Prerequisites
To perform the solution, you need to have the following prerequisites in place:

Create a Hugging Face User Access Token and get access to the gated repo mistralai/Mixtral-8x7B-v0.1 on Hugging Face.
(Optional) Create a Weights & Biases API key to access the Weights & Biases dashboard for logging and monitoring. This is recommended if you’d like to visualize model training specific metrics.
Request a service quota at Service Quotas for 1x ml.p4d.24xlarge on Amazon SageMaker. To request a service quota increase, on the AWS Service Quotas console, navigate to AWS services, Amazon SageMaker, and choose ml.p4d.24xlarge for training job usage.
Create an AWS Identity and Access Management (IAM) role with managed policies AmazonSageMakerFullAccess and AmazonEC2FullAccess to give required access to SageMaker to run the examples.

This role is for demonstration purposes only. You need to adjust it to your specific security requirements for production. Adhere to the principle of least privilege while defining IAM policies in production.

(Optional) Create an Amazon SageMaker Studio domain (see Quick setup to Amazon SageMaker) to access Jupyter notebooks with the preceding role. (You can use JupyterLab in your local setup too)
Clone the GitHub repository with the assets for this deployment. This repository consists of a notebook that references training assets.

$ git clone https://github.com/aws-samples/sagemaker-distributed-training-workshop.git
$ cd 15_mixtral_finetune_qlora

The 15_mixtral_finetune_qlora directory contains the training scripts that you might need to deploy this sample.
Next, we will run the finetune-mixtral.ipynb notebook to fine-tune the Mixtral 8x7B model using QLoRA on SageMaker. Check out the notebook for more details on each step. In the next section, we walk through the key components of the fine-tuning execution.
Solution walkthrough
To perform the solution, follow the steps in the next sections.
Step 1: Set up required libraries
Install the relevant HuggingFace and SageMaker libraries:

!pip install transformers “datasets[s3]==2.18.0” “sagemaker>=2.190.0” “py7zr” “peft==0.12.0″ –upgrade –quiet

Step 2: Load dataset
In this example, we use the GEM/viggo dataset from Hugging Face. This is a data-to-text generation dataset in the video game domain. The dataset is clean and organized with about 5,000 data points, and the responses are more conversational than information seeking. This type of dataset is ideal for extracting meaningful information from customer reviews. For example, an ecommerce application such as Amazon.com could use a similarly formatted dataset for fine-tuning a model for natural language processing (NLP) analysis to gauge interest in products sold. The results can be used for recommendation engines. Thus, this dataset is a good candidate for fine-tuning LLMs. To learn more about the viggo dataset, check out this research paper.
Load the dataset and convert it to the required prompt structure. The prompt is constructed with the following elements:

Target sentence – Think of this as the final review. In the dataset, this is target.
Meaning representation – Think of this as a deconstructed review, broken down by attributes such as inform, request, or give_opinion. In the dataset, this is meaning_representation.

Running the following cell gives us the train_set and test_set (training split and testing split, respectively) with structured prompts. We use the Python map function to structure the dataset splits according to our prompt.

def generate_and_tokenize_prompt(data_point):
full_prompt = f”””
Given a target sentence, construct the underlying
meaning representation …
[‘inform’, ‘request’, ‘give_opinion’, ‘confirm’,
‘verify_attribute’, ‘suggest’, ‘request_explanation’,
‘recommend’, ‘request_attribute’]

The attributes must be one of the following:
[‘name’, ‘exp_release_date’, ‘release_year’,
‘developer’, ‘esrb’, ‘rating’, ‘genres’,
‘player_perspective’, ‘has_multiplayer’, ‘platforms’,
‘available_on_steam’, ‘has_linux_release’,
‘has_mac_release’, ‘specifier’]

### Target sentence:
{data_point[“target”]}

### Meaning representation:
{data_point[“meaning_representation”]}
“””
return {“prompt”: full_prompt.strip()}

# Load dataset from the HuggingFace hub
train_set = load_dataset(dataset_name, split=”train”)
test_set = load_dataset(dataset_name, split=”test”)

# Add system message to each conversation
columns_to_remove = list(dataset[“train”].features)

train_dataset = train_set.map(
generate_and_tokenize_prompt,
remove_columns=columns_to_remove,
batched=False
)

test_dataset = test_set.map(
generate_and_tokenize_prompt,
remove_columns=columns_to_remove,
batched=False
)

Upload the dataset to Amazon S3. This step is crucial because the dataset stored in Amazon S3 will serve as the input data channel for the SageMaker training cluster. SageMaker will efficiently manage the process of distributing this data across the training cluster, allowing each node to access the necessary information for model training.

input_path = f’s3://{sess.default_bucket()}/datasets/mixtral’

# Save datasets to s3
train_dataset.to_json(f”{input_path}/train/dataset.json”, orient=”records”)
train_dataset_s3_path = f”{input_path}/train/dataset.json”
test_dataset.to_json(f”{input_path}/test/dataset.json”, orient=”records”)
test_dataset_s3_path = f”{input_path}/test/dataset.json”

We analyze the distribution of prompt tokens to determine the maximum sequence length required for training our model in the upcoming steps.
The following graph shows the prompt tokens plotted. The x-axis is the length of the prompts, and the y-axis is the number of times that length occurs in the training dataset (frequency). We use this to determine the maximum sequence length and pad the rest of the data points accordingly. The maximum number of words in our example is 173.

Figure 4: Graph showing the distribution of input token lengths prompted. The x-axis shows the lengths and the y-axis shows the frequency with which those input token lengths occur in the train and test dataset splits.

Step 3: Configure the parameters for SFTTrainer for the fine-tuning task
We use TrlParser to parse hyperparameters in a YAML file that is required to configure SFTTrainer API for fine-tuning the model. This approach offers flexibility because we can also overwrite the arguments specified in the config file by explicitly passing them through the command line interface.

cat > ./args.yaml <<EOF
model_id: “mistralai/Mixtral-8x7B-v0.1” # Hugging Face model id
max_seq_length: 2048 # based in prompt length distribution graph
train_dataset_path: “/opt/ml/input/data/train/” # path to where SageMaker saves train dataset
test_dataset_path: “/opt/ml/input/data/test/” # path to where SageMaker saves test dataset
output_dir: “/opt/ml/model/mixtral/adapter” # path to where SageMaker will upload the model

num_train_epochs: 1 # number of training epochs
per_device_train_batch_size: 10 # batch size per device during training
gradient_accumulation_steps: 1 # number of steps before performing a backward/update pass
optim: adamw_torch # use torch adamw optimizer

bf16: true # use bfloat16 precision
tf32: true # use tf32 precision
gradient_checkpointing: true # use gradient checkpointing to save memory

# offload FSDP parameters: https://huggingface.co/docs/transformers/main/en/fsdp
fsdp: “full_shard auto_wrap” # remove offload if enough GPU memory
fsdp_config:
backward_prefetch: “backward_pre”
forward_prefetch: “false”
use_orig_params: “false”

Step 4: Review the launch script
You are now prepared to fine-tune the model using a combination of PyTorch FSDP and QLoRA. We’ve prepared a script called launch_fsdp_qlora.py that will perform the tasks mentioned in the following steps. The following is a quick review of the key points in this script before launching the training job.

Load the dataset from a JSON file located at the specified path, using the load_dataset function to prepare it for model training.

# Load datasets
train_dataset = load_dataset(
“json”,
data_files=os.path.join(script_args.train_dataset_path,
“dataset.json”),
split=”train”,
)

Prepare the tokenizer and the model.

We employ the BitsAndBytes library to configure 4-bit quantization settings for our model, enabling memory-efficient loading and computation.
By setting parameters such as load_in_4bit and bnb_4bit_use_double_quant to True, we enable a dramatic reduction in model size without significant loss in performance. The nf4 quantization type, coupled with bfloat16 compute and storage data types, allows for nuanced control over the quantization process, striking an optimal balance between model compression and accuracy preservation. This configuration enables the deployment of massive models on resource-constrained hardware, making advanced AI more accessible and practical for a wide range of applications.

# Configure model quantization
torch_dtype = torch.bfloat16
quant_storage_dtype = torch.bfloat16

# Configures 4-bit quantization settings for the model
quantization_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_use_double_quant=True,
bnb_4bit_quant_type=”nf4″,
bnb_4bit_compute_dtype=torch_dtype,
bnb_4bit_quant_storage=quant_storage_dtype,
)

model_loading_params = {
“quantization_config”: quantization_config,
“torch_dtype”: quant_storage_dtype,
“use_cache”: False if
training_args.gradient_checkpointing else True
}

# Loads a pre-trained model from the specified model ID
model = AutoModelForCausalLM.from_pretrained(
script_args.model_id,
cache_dir=”/opt/ml/sagemaker/warmpoolcache”,
**model_loading_params
)

Initiate the training process using SFTTrainer from the Transformer Reinforcement Learning (TRL) library to fine-tune the model. The SFTTrainer simplifies the process of supervised fine-tuning for LLMs. This approach makes fine-tuning efficient to adapt pre-trained models to specific tasks or domains.

We use the LoraConfig class from the Hugging Face’s PEFT library to configure and add LoRA parameters (also called “adapters”) to the model.

# LoRA config based on QLoRA paper & Sebastian Raschka experiment
peft_config = LoraConfig(
lora_alpha=8,
lora_dropout=0.05,
r=16,

)

################
# Training
################
trainer = SFTTrainer(
model=model,
args=training_args,
train_dataset=train_dataset,
eval_dataset=test_dataset,
peft_config=peft_config,
max_seq_length=script_args.max_seq_length,
tokenizer=tokenizer,
packing=True,

)

trainer.train(resume_from_checkpoint=checkpoint)

Step 5: Fine-tune your model
To fine-tune your model, follow the steps in the next sections.
Launch the training job
You are now ready to launch the training. We use the SageMaker Training estimator, which uses torchrun to initiate distributed training.
The SageMaker estimator simplifies the training process by automating several key tasks in this example:

The SageMaker estimator spins up a training cluster of one ml.p4d.24xlarge instance. SageMaker handles the setup and management of these compute instances, which reduces your TCO.
This estimator also uses one of the pre-built containers managed by SageMaker, PyTorch, which includes an optimized compiled version of the PyTorch framework and its required dependencies and GPU-specific libraries for accelerated computations.

pytorch_estimator = PyTorch(
entry_point= ‘launch_fsdp_qlora.py’,
source_dir=”./scripts”,

framework_version=”2.2.0″,
py_version=”py310″,
instance_count=1,
instance_type=”ml.p4d.24xlarge”,
sagemaker_session=sess,
disable_output_compression=True,
keep_alive_period_in_seconds=1800,
distribution={“torch_distributed”: {“enabled”: True}},
hyperparameters={
“config”: “/opt/ml/input/data/config/args.yaml” #path to
TRL config which was uploaded to s3
}
)

The training process generates trained adapters that will be saved in a default S3 bucket named sagemaker-<region name>-<account_id> for this job.
Monitor your training run
You can monitor training metrics, such as loss, and learning rate for your training run through the Weights & Biases Dashboard. The following figures show the results of the training run, where we track GPU utilization and GPU memory utilization.
The example is optimized to use GPU memory to its maximum capacity. Note that increasing the batch size any further will lead to CUDA Out of Memory errors.
The following graph shows the GPU memory utilization (for all eight GPUs) during the training process. You can also observe the GPU memory utilization for any given point in time.

Figure 5: This graph shows the GPU Memory utilization plotted for all 8 GPUs in the training job.

The following graph shows the GPU compute utilization (for all eight GPUs) during the training process. You can also observe the GPU memory utilization for any given point in time.

Figure 6: This graph shows the GPU Compute utilization plotted for all 8 GPUs in the training job.

Step 6: Merge the trained adapter with the base model for inference
Merge the training LoRA adapter with the base model. After the merge is complete, run inference to find the results. Specifically, look at how the new fine-tuned and merged model performs compared to the original unmodified Mixtral-8x7b model. The example does the adapter merge and inference both in the same launch script “merge_model_adapter.py.”
Before launching the training job, review the key components of the merge script:
Use the Hugging Face Transformers library. Specifically, use AutoModelForCausalLM to load a PEFT model from a specified HuggingFace model directory (mistralai/Mixtral-8x7B-v0.1). We have configured this library to have a low CPU memory utilization (low_cpu_mem_usage=True) to reduce the CPU to GPU communication overhead, and we’ve also used automatic device mapping (device_map=”auto”) while offloading the model to a designated folder to manage resource constraints.

# Load a Peft model
base_model = AutoModelForCausalLM.from_pretrained(
model_id,
low_cpu_mem_usage=True,
#torch_dtype=torch.float16,
device_map=”auto”,
offload_folder=”/opt/ml/model/”
)

# Load the adapter
peft_model = PeftModel.from_pretrained(
base_model,
adapter_dir,
#torch_dtype=torch.float16,  # Set dtype to float16
offload_folder=”/opt/ml/model/”
)

# Merge the base model with the trained adapter
model = peft_model.merge_and_unload()
print(“Merge done”)

After the model is merged, send inference requests to generate responses.

def generate_text(model, prompt, max_length=500, num_return_sequences=1):

input_ids = tokenizer.encode(prompt_input,
return_tensors=”pt”).to(device)

# Generate text
with torch.no_grad():
output = model.generate(
input_ids,
max_length=max_length,
num_return_sequences=num_return_sequences,
no_repeat_ngram_size=2,
top_k=50,
top_p=0.95,
temperature=0.7
)

# Decode and return the generated text
generated_texts = [tokenizer.decode(seq,
skip_special_tokens=True) for seq in output]

return generated_texts

print(f”nnn*** Generating Inference on Base Model: {generate_text(base_model,prompt)}nnn”)

print(f”***nnn Generating Inference on Trained Model: {generate_text(model,prompt)}nnn”)

Step 7: Launch the SageMaker training job to merge the adapter
Run the following script as part of the SageMaker training job.
First, explore the adapters that were saved as part of the training run.

adapter_dir_path=f”{model_artifacts}/mixtral/adapter/”

print(f’nAdapter S3 Dir path:{adapter_dir_path} n’)

!aws s3 ls {adapter_dir_path}

# Reference Output
Adapter S3 Dir path:s3://sagemaker-<Region>-<Account-ID>/mixtral-8-7b-finetune-2024-09-08-22-27-42-099/output/model/mixtral/adapter/

PRE checkpoint-64/
PRE runs/
2024-09-08 23:08:07       5101 README.md
2024-09-08 23:07:58        722 adapter_config.json
2024-09-08 23:08:06  969174880 adapter_model.safetensors
2024-09-08 23:08:08        437 special_tokens_map.json
2024-09-08 23:08:04    1795596 tokenizer.json
2024-09-08 23:08:04        997 tokenizer_config.json
2024-09-08 23:08:04       5688 training_args.bin

Create and run the PyTorch estimator to configure the training job.

pytorch_estimator_adapter = PyTorch(
entry_point= ‘merge_model_adapter.py’,
source_dir=”./scripts”,
job_name=job_name,
base_job_name=job_name,
max_run=5800,
role=role,
framework_version=”2.2.0″,
py_version=”py310″,
instance_count=1,
instance_type=”ml.p4d.24xlarge”,
sagemaker_session=sess,
disable_output_compression=True,
keep_alive_period_in_seconds=1800,
hyperparameters={
“model_id”: “mistralai/Mixtral-8x7B-v0.1”,  # Hugging Face model id
“hf_token”: “<hf-token>”,
“dataset_name”:dataset_name
}
)

# starting the train job with our uploaded datasets as input
pytorch_estimator_adapter.fit(data, wait=True)

Here’s the target sentence (key prompt) to generate model inference results:

Earlier, you stated that you didn’t have strong feelings about PlayStation’s Little Big Adventure.
Is your opinion true for all games which don’t have multiplayer?

Ground truth inference (data label):

verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation]) 

Original model inference (that is, meaning representation):

inform(name(Little Big Adventure), has_multiplayer(Little Big Adventure))

Fine-tuned model inference result (that is, meaning representation):

verify_attribute(name[Little Big Adventure], rating[average], has_multiplayer[no], platforms[PlayStation])

The preceding results compare the inference results of the fine-tuned model against both the ground truth and the inference results of the original unmodified Mixtral 8x7B model. You can observe that the fine-tuned model provides more details and better representation of the meaning than the base model. Run systematic evaluation to quantify the fine-tuned model’s improvements for your production workloads.
Clean up
To clean up your resources to avoid incurring any more charges, follow these steps:

Delete any unused SageMaker Studio resources.
(Optional) Delete the SageMaker Studio domain.
Verify that your training job isn’t running anymore. To do so, on your SageMaker console, choose Training and check Training jobs.

Figure 7: Screenshot showing that there are no training jobs running anymore. This is what your console should look like once you follow the clean-up steps provided

To learn more about cleaning up your provisioned resources, check out Clean up.
Conclusion
In this post, we provided you with a step-by-step guide to fine-tune the Mixtral 8x7B MoE model with QLoRA. We use SageMaker Training Jobs and the Hugging Face PEFT package for QLoRA, with bitsandbytes for quantization together to perform the fine-tuning task. The fine-tuning was conducted using the quantized model loaded on a single compute instance, which eliminates the need of a larger cluster. As observed, the model performance improved with just 50 epochs.
To learn more about Mistral on AWS and to find more examples, check out the mistral-on-aws GitHub repository. To get started, check out the notebook on the mixtral_finetune_qlora GitHub repository. To learn more about generative AI on AWS, check out Generative AI on AWS, Amazon Bedrock, and Amazon SageMaker.

About the Authors
Aman Shanbhag is an Associate Specialist Solutions Architect on the ML Frameworks team at Amazon Web Services, where he helps customers and partners with deploying ML training and inference solutions at scale. Before joining AWS, Aman graduated from Rice University with degrees in computer science, mathematics, and entrepreneurship.
Kanwaljit Khurmi is an AI/ML Principal Solutions Architect at Amazon Web Services. He works with AWS product teams, engineering, and customers to provide guidance and technical assistance for improving the value of their hybrid ML solutions when using AWS. Kanwaljit specializes in helping customers with containerized and machine learning applications.
Nishant Karve is a Sr. Solutions Architect aligned with the healthcare and life sciences (HCLS) domain. He collaborates with large HCLS customers for their generative AI initiatives and guides them from ideation to production.

Amazon SageMaker Inference now supports G6e instances

As the demand for generative AI continues to grow, developers and enterprises seek more flexible, cost-effective, and powerful accelerators to meet their needs. Today, we are thrilled to announce the availability of G6e instances powered by NVIDIA’s L40S Tensor Core GPUs on Amazon SageMaker. You will have the option to provision nodes with 1, 4, and 8 L40S GPU instances, with each GPU providing 48 GB of high bandwidth memory (HBM). This launch provides organizations with the capability to use a single-node GPU instance—G6e.xlarge—to host powerful open-source foundation models such as Llama 3.2 11 B Vision, Llama 2 13 B, and Qwen 2.5 14B, offering organizations a cost-effective and high-performing option. This makes it a perfect choice for those looking to optimize costs while maintaining high performance for inference workloads.
The key highlights for G6e instances include:

Twice the GPU memory compared to G5 and G6 instances, enabling deployment of large language models in FP16 up to:

14B parameter model on a single GPU node (G6e.xlarge)
72B parameter model on a 4 GPU node (G6e.12xlarge)
90B parameter model on an 8 GPU node (G6e.48xlarge)

Up to 400 Gbps of networking throughput
Up to 384 GB GPU Memory

Use cases
G6e instances are ideal for fine-tuning and deploying open large language models (LLMs). Our benchmarks show that G6e provides higher performance and is more cost-effective compared to G5 instances, making them an ideal fit for use in low-latency, real time use cases such as:

Chatbots and conversational AI
Text generation and summarization
Image generation and vision models

We have also observed that G6e performs well for inference at high concurrency and with longer context lengths. We have provided complete benchmarks in the following section.
Performance
In the following two figures, we see that for long context length of 512 and 1024, G6e.2xlarge provides up to 37% better latency and 60% better throughput compared to G5.2xlarge for a Llama 3.1 8B model.

In the following two figures, we see that G5.2xlarge throws a CUDA out of memory (OOM) when deploying the LLama 3.2 11B Vision model, whereas G6e.2xlarge provides great performance.

In the following two figures, we compare G5.48xlarge (8 GPU node) with the G6e.12xlarge (4 GPU) node, which costs 35% less and is more performant. For higher concurrency, we see that G6e.12xlarge provides 60% lower latency and 2.5 times higher throughput.

In the below figure, we are comparing cost per 1000 tokens when deploying a Llama 3.1 70b which further highlights the cost/performance benefits of using G6e instances compared to G5.

Deployment walkthrough
Prerequisites
To try out this solution using SageMaker, you’ll need the following prerequisites:

An AWS account that will contain all of your AWS resources.
An AWS Identity and Access Management (IAM) role to access SageMaker. To learn more about how IAM works with SageMaker, see Identity and Access Management for Amazon SageMaker.
Access to Amazon SageMaker Studio or a SageMaker notebook instance, or an interactive development environment (IDE) such as PyCharm or Visual Studio Code. We recommend using SageMaker Studio for straightforward deployment and inference.
1 instance of ml.g6e.xlarge (or larger) for SageMaker hosting.

Deployment
You can clone the repository and use the notebook provided here.
Clean up
To prevent incurring unnecessary charges, it’s recommended to clean up the deployed resources when you’re done using them. You can remove the deployed model with the following code:
predictor.delete_predictor()
Conclusion
G6e instances on SageMaker unlock the ability to deploy a wide variety of open source models cost-effectively. With superior memory capacity, enhanced performance, and cost-effectiveness, these instances represent a compelling solution for organizations looking to deploy and scale their AI applications. The ability to handle larger models, support longer context lengths, and maintain high throughput makes G6e instances particularly valuable for modern AI applications. Try the code to deploy with G6e.

About the Authors
Vivek Gangasani is a Senior GenAI Specialist Solutions Architect at AWS. He helps emerging GenAI companies build innovative solutions using AWS services and accelerated compute. Currently, he is focused on developing strategies for fine-tuning and optimizing the inference performance of Large Language Models. In his free time, Vivek enjoys hiking, watching movies and trying different cuisines.
Alan Tan is a Senior Product Manager with SageMaker, leading efforts on large model inference. He’s passionate about applying machine learning to the area of analytics. Outside of work, he enjoys the outdoors.
Pavan Kumar Madduri is an Associate Solutions Architect at Amazon Web Services. He has a strong interest in designing innovative solutions in Generative AI and is passionate about helping customers harness the power of the cloud. He earned his MS in Information Technology from Arizona State University. Outside of work, he enjoys swimming and watching movies.
Michael Nguyen is a Senior Startup Solutions Architect at AWS, specializing in leveraging AI/ML to drive innovation and develop business solutions on AWS. Michael holds 12 AWS certifications and has a BS/MS in Electrical/Computer Engineering and an MBA from Penn State University, Binghamton University, and the University of Delaware.

Orchestrate generative AI workflows with Amazon Bedrock and AWS Step F …

Companies across all industries are harnessing the power of generative AI to address various use cases. Cloud providers have recognized the need to offer model inference through an API call, significantly streamlining the implementation of AI within applications. Although a single API call can address simple use cases, more complex ones may necessitate the use of multiple calls and integrations with other services.
This post discusses how to use AWS Step Functions to efficiently coordinate multi-step generative AI workflows, such as parallelizing API calls to Amazon Bedrock to quickly gather answers to lists of submitted questions. We also touch on the usage of Retrieval Augmented Generation (RAG) to optimize outputs and provide an extra layer of precision, as well as other possible integrations through Step Functions.
Introduction to Amazon Bedrock and Step Functions
Amazon Bedrock is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon through a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using Amazon Bedrock, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and Retrieval Augmented Generation (RAG), and build agents that execute tasks using your enterprise systems and data sources. Since Amazon Bedrock is serverless, you don’t have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.
AWS Step Functions is a fully managed service that makes it easier to coordinate the components of distributed applications and microservices using visual workflows. Building applications from individual components that each perform a discrete function helps you scale more easily and change applications more quickly. Step Functions is a reliable way to coordinate components and step through the functions of your application. Step Functions provides a graphical console to arrange and visualize the components of your application as a series of steps. This makes it easier to build and run multi-step applications. Step Functions automatically triggers and tracks each step and retries when there are errors, so your application executes in order and as expected. Step Functions logs the state of each step, so when things do go wrong, you can diagnose and debug problems more quickly. You can change and add steps without even writing code, so you can more easily evolve your application and innovate faster.
Orchestrating parallel tasks using the map functionality
Arrays are fundamental data structures in programming, consisting of ordered collections of elements. In the context of Step Functions, arrays play a crucial role in enabling parallel processing and efficient task orchestration. The map functionality in Step Functions uses arrays to execute multiple tasks concurrently, significantly improving performance and scalability for workflows that involve repetitive operations. Step Functions provides two different mapping strategies for iterating through arrays: inline mapping and distributed mapping, each with its own advantages and use cases.
Inline mapping
The inline map functionality allows you to perform parallel processing of array elements within a single Step Functions state machine execution. This approach is suitable when you have a relatively small number of items to process and when the processing of each item is independent of the others. Here’s how it works:

You define a Map state in your Step Functions state machine.
Step Functions iterates over the array and runs the specified tasks for each element concurrently.
The results of each iteration are collected and made available for subsequent steps in the state machine.

Inline mapping is efficient for lightweight tasks and helps avoid launching multiple Step Functions executions, which can be more costly and resource intensive. But there are limitations. When using inline mapping, only JSON payloads can be accepted as input, your workflow’s execution history can’t exceed 25,000 entries, and you can’t run more than 40 concurrent map iterations.
Distributed mapping
The distributed map functionality is designed for scenarios where many items need to be processed or when the processing of each item is resource intensive or time-consuming. Instead of handling all items within a single execution, Step Functions launches a separate execution for each item in the array, letting you concurrently process large-scale data sources stored in Amazon Simple Storage Service (Amazon S3), such as a single JSON or CSV file containing large amounts of data, or even a large set of Amazon S3 objects. This approach offers the following advantages:

Scalability – By distributing the processing across multiple executions, you can scale more efficiently and take advantage of the built-in parallelism in Step Functions
Fault isolation – If one execution fails, it doesn’t affect the others, providing better fault tolerance and reliability
Resource management – Each execution can be allocated its own resources, helping prevent resource contention and providing consistent performance

However, distributed mapping can incur additional costs due to the overhead of launching multiple Step Functions executions.
Choosing a mapping approach
In summary, inline mapping is suitable for lightweight tasks with a relatively small number of items, whereas distributed mapping is better suited for resource-intensive tasks or large datasets that require better scalability and fault isolation. The choice between the two mapping strategies depends on the specific requirements of your application, such as the number of items, the complexity of processing, and the desired level of parallelism and fault tolerance.
Another important consideration when building generative AI applications using Amazon Bedrock and Step Functions Map states together would be the Amazon Bedrock runtime quotas. Generally, these model quotas allow for hundreds or even thousands of requests per minute. However, you may run into issues trying to run a large map on models with low requests processed per minute quotas, such as image generation models. In that scenario, you can include a retrier in the error handling of your Map state.
Solution overview
In the following sections, we get hands-on to see how this solution works. Amazon Bedrock has a variety of model choices to address specific needs of individual use cases. For the purposes of this exercise, we use Amazon Bedrock to run inference on Anthropic’s Claude 3.5 Haiku model to receive answers to an array of questions because it’s a performant, fast, and cost-effective option.
Our goal is to create an express state machine in Step Functions using the inline Map state to parse through the JSON array of questions sent by an API call from an application. For each question, Step Functions will scale out horizontally, creating a simultaneous call to Amazon Bedrock. After all the answers come back, Step Functions will concatenate them into a single response, which our original calling application can then use for further processing or displaying to end-users.
The payload we send consists of an array of nine Request for Proposal (RFP) questions, as well as a company description:

{
“questions”: [
“Can you describe your technical capabilities and infrastructure?”,
“What security measures do you have in place to protect data and privacy?”,
“Can you provide case studies or examples of similar projects you have handled?”,
“How do you handle project management, and what tools do you use?”,
“What are your support and maintenance services like?”,
“What is your pricing model?”,
“Can you provide references from other clients?”,
“How do you ensure the scalability of your solution?”,
“What is your approach to data backup and recovery?”
],
“description”: “Our company, AnyCompany Tech, boasts a robust technical infrastructure that allows us to handle complex projects with ease. Our strength lies in our dynamic team of experts and our cutting-edge technology, which, when combined, can deliver solutions of any scale. We’ve worked with clients across the globe, for instance, our project with Example Corp involved a sophisticated upgrade of their system. In terms of security, we prioritize data privacy and have put in place stringent measures to ensure that all data is stored securely. We’re quite proud of our project with AnyCompany Networks, where we overhauled their security systems to bolster their data protection capabilities. We use a range of project management tools, including Product-1 and Product-2, which allows us to customize our approach to each client’s needs. Our pricing model varies depending on the project, but we always aim to provide cost-effective solutions. We’ve had numerous positive feedback from our clients, with Example Corp and AnyCompany Networks among those who have expressed satisfaction with our services. We’re more than happy to provide further references upon request. Software updates and upgrades are a critical part of our service. We have a dedicated team that ensures all systems are up-to-date and running smoothly. Furthermore, our solutions are designed to be scalable, ensuring that they can grow alongside your business. Lastly, in terms of data backup and recovery, we have a comprehensive plan in place, which includes regular data backups and a robust recovery strategy. We understand the importance of data in today’s world and we’re committed to ensuring its safety and accessibility at all times.”
}

You can use the step-by-step guide in this post or use the prebuilt AWS CloudFormation template in the us-west-2 Region to provision the necessary AWS resources. AWS CloudFormation gives developers and businesses a straightforward way to create a collection of related AWS and third-party resources, and provision and manage them in an orderly and predictable fashion.
Prerequisites
You need the following prerequisites to follow along with this solution implementation:

An AWS account
An AWS Identity and Access Management (IAM) user that has access to Amazon Bedrock and Step Functions
Model access to Anthropic Claude 3.5 Haiku on Amazon Bedrock in a supported AWS Region (we will use us-west-2)

Create a State Machine and add a Map state
In the AWS console in the us-west-2 Region, launch into Step Functions, and select Get started and Create your own to open a blank canvas in Step Functions Workflow Studio.
Edit the state machine by adding an inline Map state with items sourced from a JSON payload.

Next, tell the Map state where the array of questions is located by selecting Provide a path to items array and pointing it to the questions array using JSONPath syntax. Selecting Modify items with ItemSelector allows you to structure the payload, which is then sent to each of the child workflow executions. Here, we map the description through with no change and use $$.Map.Item.Value to map the question from the array at the index of the map iteration.

Invoke an Amazon Bedrock model
Next, add a Bedrock: InvokeModel action task as the next state within the Map state.

Now you can structure your Amazon Bedrock API calls through Workflow Studio. Because we’re using Anthropic’s Claude 3.5 Haiku model on Amazon Bedrock, we select the corresponding model ID for Bedrock model identifier and edit the provided sample with instructions to incorporate the incoming payload. Depending on which model you select, the payload may have a different structure and prompt syntax.

Build the payload
The prompt you build uses the Amazon State Language intrinsic function States.Format in order to do string interpolation, substituting {} for the variables declared after the string. We must also include .$ after our text key to reference a node in this state’s JSON input.
When building out this prompt, you should be very prescriptive in asking the model to do the following:

Answer the questions thoroughly using the following description
Not repeat the question
Only respond with the answer to the question

We set the max_tokens to 800 to allow for longer responses from Amazon Bedrock. Additionally, you can include other inference parameters such as temperature, top_p, top_k, and stop_sequences. Tuning these parameters can help limit the length or influence the randomness or diversity of the model’s response. For the sake of this example, we keep all other optional parameters as default.

{
“anthropic_version”: “bedrock-2023-05-31”,
“max_tokens”: 800,
“messages”: [
{
“role”: “user”,
“content”: [
{
“type”: “text”,
“text.$”: “States.Format(‘Answer following question thoroughly, using the following description. Do not repeat the question. Only respond with the answer to the question. Question: {} Description: {}’, $.questions.question, $.description)”
}
]
}
]
}

Form the response
To provide a cleaner response back to our calling application, we want to use some options to transform the output of the Amazon Bedrock Task state. First, use ResultSelector to filter the response coming back from the service to pull out the text completion, then add the original input back to the output using ResultPath and finish by filtering the final output using OutputPath. That way you don’t have to see the description being mapped unnecessarily for each array item.

To simulate the state machine being called by an API, choose Execute in Workflow Studio. Using the preceding input, the Step Functions output should look like the following code, although it may vary slightly due to the diversity and randomness of FMs:

{
“output”: [
{
“question”: “Can you describe your technical capabilities and infrastructure?”,
“response”: {
“completion”: “AnyCompany Tech offers a comprehensive technical infrastructure powered by a dynamic expert team and cutting-edge technology capable of handling complex projects at any scale. The company prioritizes data security through stringent protection measures, uses advanced project management tools like Product-1 and Product-2, and offers customizable solutions. They provide global client support with a focus on scalability, software updates, and robust data backup and recovery strategies. Their approach emphasizes cost-effectiveness, client satisfaction, and adaptable technological solutions that can grow with a client’s business needs. The team maintains a dedicated focus on keeping systems updated, secure, and responsive to evolving technological requirements.”
}
},
{
“question”: “What security measures do you have in place to protect data and privacy?”,
“response”: {
“completion”: “At AnyCompany Tech, we have implemented stringent security measures to protect data and privacy. These include a comprehensive data backup and recovery strategy with regular backups, robust security systems that have been successfully deployed for clients like AnyCompany Networks, and dedicated teams ensuring systems are up-to-date. We prioritize data privacy and use advanced technical infrastructure and cutting-edge technology to secure client information. Our approach involves customizable security solutions, scalable systems, and a commitment to protecting data accessibility and safety at all times.”
}
},
{
“question”: “Can you provide case studies or examples of similar projects you have handled?”,
“response”: {
“completion”: “Based on the description provided, we have successfully completed projects for Example Corp, involving a sophisticated system upgrade, and AnyCompany Networks, where we overhauled their security systems. These case studies demonstrate our ability to handle complex technical infrastructure projects across different industries. Our team has worked with global clients, delivering solutions that prioritize security, data privacy, and scalability. While specific technical details aren’t elaborated, our track record shows proven expertise in system improvements and security enhancements. We have received positive client feedback from both Example Corp and AnyCompany Networks, and we are prepared to provide additional references upon request.”
}
},
{
“question”: “How do you handle project management, and what tools do you use?”,
“response”: {
“completion”: “At AnyCompany Tech, we utilize Product-1 and Product-2 as our primary project management tools, allowing us to create customized approaches for each client’s unique needs. Our project management strategy is built on our dynamic team of experts and cutting-edge technology, which enables us to handle complex projects efficiently. We prioritize adaptability, scalability, and client-specific solutions, ensuring that our project management approach can accommodate projects of any scale. Our tools and processes are designed to provide transparent tracking, seamless communication, and effective resource allocation, which allows us to deliver high-quality results consistently. Additionally, we have dedicated teams for software updates, system maintenance, and data backup/recovery to support our robust project management infrastructure.”
}
},
{
“question”: “What are your support and maintenance services like?”,
“response”: {
“completion”: “Our support and maintenance services are comprehensive and tailored to each client’s unique needs. We provide a dedicated team that ensures all systems are up-to-date and running smoothly, with regular software updates and upgrades. Our solutions are designed to be scalable, allowing them to grow alongside your business. We have a robust data backup and recovery strategy that includes regular backups and a comprehensive recovery plan to ensure data safety and accessibility. Our technical infrastructure, combined with our expert team, allows us to deliver high-quality, secure, and efficient maintenance services. We use advanced project management tools like Product-1 and Product-2 to customize our approach, and we prioritize data privacy with stringent security measures. Our goal is to provide cost-effective solutions that meet and exceed client expectations, with a track record of successful projects and satisfied clients like Example Corp and AnyCompany Networks.”
}
},
{
“question”: “What is your pricing model?”,
“response”: {
“completion”: “Our pricing model is flexible and tailored to each specific project. We aim to provide cost-effective solutions that are customized to meet the unique needs of our clients. The exact pricing depends on the scope, complexity, and requirements of the project, ensuring that we deliver value while addressing the specific challenges of each client’s technological needs.”
}
},
{
“question”: “Can you provide references from other clients?”,
“response”: {
“completion”: “Yes, we can provide references from clients like Example Corp and AnyCompany Networks, who have previously worked with us and expressed satisfaction with our services. We are more than happy to connect you with these references upon request, allowing you to gain direct insights into our capabilities, project performance, and customer satisfaction.”
}
},
{
“question”: “How do you ensure the scalability of your solution?”,
“response”: {
“completion”: “We ensure scalability through a multi-faceted approach. Our solutions are intentionally designed to be flexible and adaptable, capable of growing alongside our clients’ businesses. We leverage cutting-edge technology and our dynamic team of experts to create infrastructure that can handle increasingly complex projects. Our robust technical infrastructure allows us to seamlessly expand systems, and we use advanced project management tools like Product-1 and Product-2 to customize solutions that can scale effectively. Additionally, we have a dedicated team focused on software updates and upgrades, which helps maintain system performance and enables smooth expansion. Our comprehensive approach to scalability is further supported by our data backup and recovery strategies, ensuring system reliability and continuity as businesses grow. By prioritizing adaptability and having the technical expertise to implement scalable solutions, we can confidently meet the evolving needs of our clients across various industries.”
}
},
{
“question”: “What is your approach to data backup and recovery?”,
“response”: {
“completion”: “Our data backup and recovery approach encompasses regular, comprehensive data backups and a robust recovery strategy. We understand the critical importance of data protection and have implemented a systematic process to ensure data safety and accessibility at all times. Our strategy involves creating multiple backup copies, storing them in secure, redundant locations, and maintaining a well-defined recovery protocol that allows for quick restoration of data in the event of any potential loss or system failure. This approach is designed to minimize downtime and protect our clients’ valuable information, reflecting our commitment to maintaining the highest standards of data security and reliability.”
}
}
],
“outputDetails”: {
“truncated”: false
}
}

Clean up resources
To delete this solution, navigate to the State machines page on the Step Functions console, select your state machine, choose Delete, and enter delete to confirm. It will be marked for deletion and will be deleted when all executions are stopped.
RAG and other possible integrations
RAG is a strategy that enhances the output of a large language model (LLM) by allowing it to reference an authoritative external knowledge base, generating more accurate or secure responses. This powerful tool can extend the capabilities of LLMs to specific domains or an organization’s internal knowledge base without needing to retrain or even fine-tune the model.
A straightforward way to integrate RAG into the preceding RFP example is by adding a Bedrock Runtime Agents: Retrieve action task to your Map state before invoking the model. This enables queries to Amazon Bedrock Knowledge Bases, which supports various vector storage databases, including the Amazon OpenSearch Serverless vector engine, Pinecone, Redis Enterprise Cloud, and soon Amazon Aurora and MongoDB. Using Knowledge Bases to ingest and vectorize example RFPs and documents stored in Amazon S3 eliminates the need to include a description with the question array. Also, because a vector store can accommodate a broader range of information than a single prompt is able to, RAG can greatly enhance the specificity of the responses.
In addition to Amazon Bedrock Knowledge Bases, there are other options to integrate for RAG depending on your existing tech stack, such as directly with an Amazon Kendra Task state or with a vector database of your choosing through third-party APIs using HTTP Task states.

Step Functions offers composability, allowing you to seamlessly integrate over 9,000 AWS API actions from more than 200 services directly into your workflows. These optimized service integrations simplify the use of common services like AWS Lambda, Amazon Elastic Container Service (Amazon ECS), AWS Glue, and Amazon EMR, offering features such as IAM policy generation and the Run A Job (.sync) pattern, which automatically waits for the completion of asynchronous jobs. Another common pattern seen in generative AI applications is chaining models together to accomplish secondary tasks, like language translation after a primary summarization task is completed. This can be accomplished by adding another Bedrock: InvokeModel action task just as we did earlier.
Conclusion
In this post, we demonstrated the power and flexibility of Step Functions for orchestrating parallel calls to Amazon Bedrock. We explored two mapping strategies—inline and distributed—for processing small and large datasets, respectively. Additionally, we delved into a practical use case of answering a list of RFP questions, demonstrating how Step Functions can efficiently scale out and manage multiple Amazon Bedrock calls.
We introduced the concept of RAG as a strategy for enhancing the output of an LLM by referencing an external knowledge base and demonstrated multiple ways to incorporate RAG into Step Functions state machines. We also highlighted the integration capabilities of Step Functions, particularly the ability to invoke over 9,000 AWS API actions from more than 200 services directly from your workflow.
As next steps, explore the possibilities of application patterns offered by the GenAI Quick Start PoCs GitHub repo as well as various Step Functions integrations through sample project templates within Workflow Studio. Also, consider integrating RAG into your workflows to use your organization’s internal knowledge base or specific domain expertise.

About the Author
Dimitri Restaino is a Brooklyn-based AWS Solutions Architect specialized in designing innovative and efficient solutions for healthcare companies, with a focus on the potential applications of AI, blockchain and other promising industry disruptors. Off the clock, he can be found spending time in nature or setting fastest laps in his racing sim.

Email Marketing Tools: 25 Powerful Platforms to Level Up Engagement in …

Did you know you can get a free email deliverability audit from Customers.ai? Start your free trial and get an automatic deliverability score!

So, you want to grow your business and need email marketing tools that are easy to use, affordable, and will turn your leads into customers, correct?

Lucky for you, we’ve narrowed down the more than 500 email marketing tools on the market to make up the best suite of email marketing tools from email capture software to inbox delivery.

Email marketing may be one of the oldest, non-sexy communication methods. However, it’s still highly effective and of paramount importance.

In fact, according to Emailmonday, for every $1 spent on email marketing, a return of $38 is generated on average. That’s an ROI that can’t be ignored and demands the careful selection of tools to obtain the same or better results.

We’ve broken down the 25 best email marketing tools:

Customers.ai: Email Capture Tool

MailChimp: Email Newsletter Tool 

HubSpot: Email Marketing CRM Tool  

Woodpecker: Cold Email Software

Autopilot: Email Marketing Automation Tool

Spently: Email Marketing Tool for Ecommerce 

Mailgun: Best Transactional Email Service  

Leadpresso: Email Marketing Tool for Digital Agencies and SaaS Companies  

AWeber: Email Marketing Software for Small Business 

Jasper: Email Marketing Copywriting Tool

Gmail Ads: Email Advertising Tools 

SendGrid: Email Marketing Platform

Prospect.io: Email Lookup Tool 

Mysignature.io: Email Signature Tool  

Consumer Directory: Email List Builder

Klaviyo: Email Marketing Automation 

Phantombuster: Best Email Scraper

Sendlane: Email Marketing Automation

Email Octopus: Email Marketing for SMB

Mail Genius: Email Spam Tester 

Warmy.io: Email Warmup Tool 

Email Validator & Anti-Spam Too 

GlockApps: Email Deliverability Checker

Omnisend: Email Subject Line Checker 

MailCharts: Email Competitive Intelligence

Convert Website Visitors into Real Contacts!

Identify who is visiting your site with name, email and more. Get 500 contacts for free!

Please enable JavaScript in your browser to complete this form.Website / URL *View my free report

How to Choose the Right Email Marketing Tool for Your Business

Before we get into the tools themselves, we have to talk about how to choose them. Because honestly, with so many tools at your disposal, picking the right email marketing tool can feel overwhelming.

But here’s the deal: the “best” tool isn’t one-size-fits-all. It’s about finding the one that fits your business size, goals, and budget.

Let’s break it down:

1. Assess Your Business Size

Are you running a small side hustle or managing a fast-growing brand? If you’re just starting, look for tools with free plans or low-cost tiers that still pack a punch, like Mailchimp or MailerLite. Fun fact: 62% of small businesses use email marketing as their main communication channel—so it’s worth investing in the right tool early.

2. Define Your Goals

What’s the main reason you’re using email marketing?

Building a list? Focus on tools with strong lead generation features like pop-ups and sign-up forms.

Driving ecommerce sales? Look for platforms with robust automation and segmentation, like Klaviyo or Omnisend.

Boosting engagement? Choose a tool with advanced personalization and analytics to fine-tune your campaigns.

3. Match Features to Your Needs

Not every business needs all the bells and whistles. Make a checklist of must-haves, like automation, drag-and-drop design, A/B testing, or integration with tools you’re already using.

4. Consider Your Budget

Email marketing doesn’t have to break the bank. Many tools offer tiered pricing, so you’re only paying for what you need. If you’re just starting, look for free plans that include essentials, but be ready to scale up when your email list grows.

5. Test Before You Commit

Most platforms offer free trials—use them! Test out the interface, see how easy it is to set up campaigns, and make sure it fits your workflow.

Remember that choosing the right email marketing tool isn’t just about features. It’s about finding a solution that works for where you are now AND where you’re headed.

Must-Have Features in Modern Email Marketing Tools

We know that every email marketing tool is different. Some are sleek and powerful while others feel like they were designed in 2005 and never updated. If you want to nail your email game, these are the features you absolutely can’t live without:

1. Automation That Actually Feels Human

Automation isn’t just about saving time, it’s about staying relevant. The best tools let you set up smart workflows, like sending a cart reminder two days after someone ghosts on checkout.

Bonus points if the emails don’t scream “robot wrote this.”

2. Analytics That Go Beyond Opens and Clicks

You need more than “20 people opened your email.” Look for tools that show you real ROI — like how much cash your last campaign brought in!

The great ones even let you dig into customer journeys so you can see where people drop off and where they buy.

3. Segmentation That’s Not a Headache

If your tool doesn’t let you easily slice and dice your audience, skip it. Want to send a “we miss you” email to people who haven’t bought in 60 days? Or reward your VIP customers?

Good segmentation makes personalization easy and makes you look like a marketing genius.

4. Templates That Make You Look Like a Designer

We’re not all Canva experts and that’s okay. Modern tools should come with drag-and-drop editors and stunning templates you can tweak in minutes.

Have you seen our new drag and drop email editor? Our partner lilikoi agency used it to set up an on-brand welcome email to high-intent visitors for our customer Solaris Solar. The result? A 37.5% open rate and 9.9% click rate pic.twitter.com/PIEFMHYsdD— CustomersAI (@CustomersAI) November 22, 2024

Because no one has time for ugly emails.

5. Integrations That Play Nice

Your email tool needs to get along with the rest of your stack. Whether it’s syncing with Shopify, your CRM, or even your SMS marketing app. The less manual data entry, the better.

6. Personalization Without the Cheesy Feel

“Hi [First Name]” is fine, but real personalization goes deeper. Think product recommendations based on what they actually browsed or “we saved this for you” emails. The best tools make this easy and, most importantly, not cringe.

7. Spam Law Savvy

You know those unsubscribe links that feel impossible to find? Don’t be that brand. Great tools make compliance with GDPR, CCPA, and CAN-SPAM painless, so you can focus on connecting with customers, not worrying about fines.

If your email tool doesn’t check these boxes, it’s time for an upgrade. The right features can turn your emails into conversions, and your audience will thank you for it (with clicks and cash).

The 25 Email Marketing Tools You Need to Know

When it comes to email marketing, the right tools can make all the difference. Whether you’re building your first list, optimizing campaigns, or scaling up to reach a bigger audience, these tools are here to help.

From beginner-friendly platforms to advanced solutions for seasoned pros, here’s a look at the best email marketing tools for every stage of your journey. Let’s dive in!

Note: Pricing and ratings as of November 2024

#1. Customers.ai: Email Capture Tool

Did you know you can capture the email addresses of people who visit your website without email forms?

With website visitor identification, you can identify ~20% of your anonymous traffic. 

That means you can capture emails simply by having people land on your website.

All you need to do is install one snippet of code on your site.

Here’s how to capture emails with Customers.ai:

1. Sign up for a free account

If you don’t already have a Customers.ai account, sign up here (no credit card is required) and connect your business.

2. Install the x-ray pixel on your site

Installing the website identification x-ray pixel is easy and can be done through Tag Manager, Shopify, WordPress, and more

3. Verify the x-ray pixel is firing

4. Start capturing emails

That’s it! Once the pixel is installed and verified, you can start capturing emails from your website visitors.

You can use the data you capture for:

Email marketing and retargeting

Creating Facebook ad lookalike audiences

Triggering direct mail to be sent

SMS campaigns

#2. MailChimp: Email Newsletter Tool 

Best for: Small businesses looking for an easy-to-use, free email marketing tool.

What It Does:MailChimp is one of the most well-known email marketing platforms, ideal for creating and sending newsletters, automating basic campaigns, and tracking performance. It’s particularly popular with small businesses due to its free plan and beginner-friendly interface. With MailChimp, you can build campaigns that look professional without needing design or coding skills, making it a top choice for those just starting out.

Standout Features:

Drag-and-Drop Builder: Create polished, professional-looking emails with ease.

Basic Automation: Set up welcome emails, abandoned cart reminders, and follow-ups with simple workflows.

Audience Segmentation: Segment your contacts based on behavior, demographics, or engagement to send targeted emails.

Templates Library: Access a variety of pre-designed templates for different industries and goals.

Free Plan: Includes essential features like email campaigns and basic analytics for up to 500 contacts.

Pricing:

Free Plan: For up to 500 contacts and 1,000 monthly sends.

Paid Plans: Start at $13/month with additional features like advanced automation and A/B testing.

Rating:★★★★☆ (4.5/5)

#3. HubSpot: Email Marketing CRM Tool  

Best for: Businesses seeking an integrated CRM and email marketing solution.

What It Does: HubSpot offers a comprehensive platform that combines email marketing with robust CRM capabilities. This integration allows businesses to manage contacts, personalize emails, and automate campaigns seamlessly. HubSpot’s user-friendly interface and powerful features make it suitable for both small businesses and enterprises aiming to enhance their marketing efforts.

Standout Features:

CRM Integration: Access detailed contact information and interaction history to tailor your email campaigns effectively.

Drag-and-Drop Email Builder: Design professional emails effortlessly with customizable templates.

Automation Workflows: Automate email sequences based on user behavior and engagement.

Analytics and Reporting: Gain insights into email performance, including open rates, click-through rates, and conversions.

Segmentation: Create targeted lists based on various criteria to send personalized content.

Pricing:

Free Plan: Includes basic CRM and email marketing features.

Paid Plans: Start at $50/month, offering advanced functionalities like A/B testing and enhanced automation.

Rating: ★★★★☆ (4.7/5)

#4. Woodpecker: Cold Email Software

Best for: B2B companies and agencies seeking personalized cold email outreach with automation.

What It Does: Woodpecker is a cold email software designed to help businesses automate personalized email campaigns and follow-ups. It focuses on improving deliverability and engagement by mimicking human-like sending behavior, making it ideal for sales teams and agencies aiming to connect with prospects effectively.

Standout Features:

Personalized Email Sequences: Create tailored email sequences with custom fields to address recipients individually.

Automated Follow-Ups: Set up automatic follow-up emails that stop when a reply is received, ensuring timely communication.

Deliverability Optimization: Features like email warm-up and bounce detection help maintain a healthy sender reputation.

A/B Testing: Test different email variations to determine the most effective messaging.

Integrations: Seamlessly connect with CRM systems and other tools to streamline workflows.

Pricing:

Starting at $49 per month per slot: Includes core features suitable for small teams.

Custom Plans: Available for larger teams and agencies with additional requirements.

Rating: ★★★★☆ (4.6/5)

#5. Autopilot: Email Marketing Automation Tool

Best for: Businesses seeking a visual approach to email marketing automation.

What It Does: Autopilot is a marketing automation platform that enables businesses to design, execute, and optimize customer journeys through a visual interface. It focuses on simplifying the automation process, making it accessible for teams without extensive technical expertise.

Standout Features:

Visual Journey Builder: Create and manage customer journeys using a drag-and-drop interface, allowing for intuitive workflow design.

Multi-Channel Marketing: Engage customers across various channels, including email, SMS, and in-app messaging, ensuring a cohesive experience.

Personalization: Deliver tailored content based on customer behavior and preferences to enhance engagement.

Integration Capabilities: Connect seamlessly with CRM systems, analytics tools, and other marketing platforms to streamline operations.

Analytics and Reporting: Access detailed insights into campaign performance to inform data-driven decisions.

Pricing:

Starting at $49 per month: Suitable for up to 2,000 contacts, with scalable options available as your business grows.

Rating: ★★★★☆ (4.5/5)

#6. UpOrder: Email Marketing Tool for Ecommerce 

Best for: Shopify store owners aiming to enhance transactional emails to boost customer engagement and repeat purchases.

What It Does: UpOrder is a Shopify app designed to transform standard transactional emails into powerful marketing tools. By customizing order confirmations, shipping updates, and other notifications, UpOrder helps ecommerce businesses drive additional sales and foster customer loyalty.

Standout Features:

Customizable Email Templates: Easily tailor transactional emails to reflect your brand’s identity, ensuring a consistent customer experience.

Product Recommendations: Include personalized product suggestions within emails to encourage repeat purchases.

Dynamic Discounts: Incorporate unique discount codes in emails to incentivize future orders.

Analytics Dashboard: Monitor email performance metrics such as open rates, click-through rates, and revenue generated to optimize campaigns.

Seamless Shopify Integration: Designed specifically for Shopify, ensuring smooth setup and operation without the need for coding skills.

Pricing:

Free Plan: Covers up to 25 orders per month.

Paid Plans: Start at $10 per month for every additional 50 orders, with customized plans available for higher volumes.

Rating: ★★★★☆ (4.8/5)

#7. Mailgun: Best Transactional Email Service  

Best for: Developers and businesses seeking a robust transactional email service with high deliverability and scalability.

What It Does: Mailgun is a powerful email delivery service designed for sending, receiving, and tracking transactional emails. It offers a suite of APIs that enable developers to seamlessly integrate email functionalities into their applications, ensuring reliable and efficient email communication.

Standout Features:

Transactional Email API: Send and receive emails effortlessly using Mailgun’s flexible and scalable API.

SMTP Relay: Utilize Mailgun’s SMTP service for sending emails, providing compatibility with various email clients and applications.

Email Validation: Improve deliverability by validating email addresses to reduce bounces and maintain a clean mailing list.

Analytics and Reporting: Access detailed insights into email performance, including delivery status, open rates, and click-through rates.

Deliverability Optimization: Benefit from tools and services aimed at enhancing email deliverability and inbox placement.

Pricing:

Free Plan: Includes 5,000 emails for the first three months.

Paid Plans: Start at $35 per month, offering additional features and higher email volumes.

Rating: ★★★★☆ (4.6/5)

#8. Leadpresso: Email Marketing Tool for Digital Agencies and SaaS Companies  

Best for: Digital agencies and B2B SaaS companies seeking to enhance lead generation and outreach efforts.

What It Does: Leadpresso is a lead generation platform designed to help businesses identify and connect with potential clients. By leveraging a vast database enriched with data from over 50 sources, Leadpresso enables users to discover companies that align with their target audience and initiate personalized outreach campaigns.

Standout Features:

Extensive Database: Access a comprehensive collection of businesses, particularly those with active Facebook Business Pages, providing valuable insights into their online presence.

Data Enrichment: Utilize information from multiple sources, including company websites and social networks, to gain a holistic view of potential leads.

Advertiser Search: Identify companies already investing in online advertising, indicating a readiness to engage in marketing services.

Personalized Outreach: Obtain contact information such as emails and phone numbers to craft tailored marketing campaigns at scale.

User-Friendly Interface: Navigate the platform with ease, allowing for efficient lead discovery and management.

Pricing:

Subscription Plans: Leadpresso offers various pricing tiers to accommodate different business needs, with details available upon request.

Rating: ★★★★☆ (4.5/5)

#9. AWeber: Email Marketing Software for Small Business 

Best for: Small businesses and entrepreneurs seeking a user-friendly email marketing solution with robust automation and support.

What It Does: AWeber is an email marketing platform designed to help small businesses create and send engaging emails, automate campaigns, and grow their subscriber lists. With a focus on simplicity and effectiveness, AWeber offers a range of tools to design professional emails, build landing pages, and analyze campaign performance.

Standout Features:

Drag-and-Drop Email Builder: Create visually appealing emails without any coding skills using customizable templates.

Email Automation: Set up automated email sequences to nurture leads and engage customers based on their actions.

Landing Page Builder: Design and publish landing pages to capture leads and promote products or services.

Subscriber Management: Organize and segment your audience for targeted email campaigns.

Analytics and Reporting: Track email performance metrics such as open rates, click-through rates, and conversions to optimize your strategy.

24/7 Customer Support: Access support via phone, live chat, or email to assist with any questions or issues.

Pricing:

Free Plan: Available for up to 500 subscribers and 3,000 emails per month, including essential features.

Paid Plans: Start at $12.50 per month, offering additional features and higher subscriber limits.

Rating: ★★★★☆ (4.6/5)

#10. Jasper: Email Marketing Copywriting Tool

Best for: Marketers and businesses aiming to enhance their email campaigns with AI-generated, on-brand content.

What It Does: Jasper is an AI-powered content generation platform designed to assist in crafting compelling email copy. By leveraging advanced language models, Jasper helps users create engaging subject lines, personalized messages, and persuasive calls-to-action, streamlining the email marketing process.

Standout Features:

AI-Generated Content: Produce high-quality email copy tailored to your brand’s voice and audience preferences.

Customizable Templates: Access a variety of templates for different email types, including newsletters, promotions, and follow-ups.

Tone and Style Adjustment: Modify the tone of your emails to match your brand’s personality, whether it’s formal, casual, or witty.

Integration Capabilities: Seamlessly integrate with popular email marketing platforms to streamline your workflow.

Continuous Learning: Benefit from Jasper’s ability to learn from your inputs, improving content suggestions over time.

Pricing:

Starter Plan: Begins at $29 per month, suitable for individuals and small teams.

Boss Mode: Starts at $59 per month, offering advanced features and increased word limits.

Business Plan: Custom pricing available for larger organizations with specific needs.

Rating: ★★★★☆ (4.7/5)

#11. Gmail Ads: Email Advertising Tools 

Best for: Businesses aiming to reach potential customers directly within their Gmail inboxes through targeted advertising.

What It Does: Gmail Ads are interactive advertisements that appear within users’ Gmail accounts, typically in the Promotions or Social tabs. These ads start as collapsed messages and expand upon clicking, resembling standard emails. They allow businesses to engage users in a familiar environment, promoting products, services, or content directly in their inboxes.

Standout Features:

Native Email Integration: Ads appear seamlessly within the Gmail interface, increasing the likelihood of user engagement.

Targeting Options: Utilize Google’s robust targeting capabilities, including demographics, interests, and behaviors, to reach specific audiences.

Cost-Effective Pricing: Operate on a pay-per-click (PPC) model, ensuring you only pay when a user interacts with your ad.

Customizable Ad Formats: Design ads that include images, videos, and call-to-action buttons to enhance user interaction.

Performance Tracking: Access detailed analytics to monitor ad performance and optimize campaigns accordingly.

Pricing:

Flexible Budgeting: Set daily budgets and bids according to your campaign goals, with no minimum spend requirement.

Rating: ★★★★☆ (4.5/5)

#12. SendGrid: Email Marketing Platform 

Best for: Businesses of all sizes seeking a reliable platform for both transactional and marketing emails.

What It Does: SendGrid is a cloud-based email delivery service that enables businesses to send transactional and marketing emails at scale. It offers robust APIs for seamless integration, ensuring high deliverability rates and comprehensive analytics to monitor email performance.

Standout Features:

Email API: Integrate email sending capabilities into your applications with ease, supporting both SMTP and RESTful APIs.

Marketing Campaigns: Design and send engaging marketing emails using a user-friendly interface with drag-and-drop functionality.

Deliverability Optimization: Benefit from tools and expertise aimed at maximizing inbox placement and engagement rates.

Analytics and Reporting: Access detailed insights into email performance, including open rates, click-through rates, and delivery statistics.

Scalability: Handle high-volume email sending, processing billions of emails monthly for businesses worldwide.

Pricing:

Free Plan: Includes 100 emails per day with essential features.

Paid Plans: Start at $19.95 per month, offering higher sending limits and advanced functionalities.

Rating: ★★★★☆ (4.6/5)

#13. Overloop: Email Lookup Tool 

Best for: Sales and marketing professionals seeking to efficiently find and verify business email addresses for outreach campaigns.

What It Does: Overloop is an email lookup tool designed to help users discover and verify professional email addresses. By providing the first name, last name, and company domain, Overloop’s Email Finder searches for the correct email address, facilitating effective prospecting and outreach efforts.

Standout Features:

Email Finder: Locate prospects’ email addresses by inputting their name and company domain, streamlining the lead generation process.

Chrome Extension: Find emails directly from LinkedIn profiles or company websites, enhancing efficiency during prospecting.

Bulk Search: Upload a list of prospects to find multiple email addresses simultaneously, saving time on large campaigns.

Email Verification: Ensure the validity of email addresses to maintain high deliverability rates and reduce bounce rates.

Integration Capabilities: Connect with CRM systems and other marketing tools to seamlessly incorporate email data into existing workflows.

Pricing:

Free Plan: Offers a limited number of email searches per month.

Paid Plans: Start at $39 per month, providing additional searches and advanced features.

Rating: ★★★★☆ (4.5/5)

#14. Mysignature.io: Email Signature Tool 

Best for: Professionals and businesses aiming to create and manage personalized, professional email signatures across various email clients.

What It Does: MySignature is an online platform that enables users to design and implement customized email signatures. It offers a range of templates and design tools, ensuring consistency and professionalism in email communications. The platform supports major email clients, including Gmail, Outlook, Apple Mail, and Yahoo Mail.

Standout Features:

User-Friendly Editor: Create and customize email signatures with an intuitive drag-and-drop interface.

Responsive Templates: Access a variety of professionally designed templates that adapt to different devices and screen sizes.

Social Media Integration: Include social media icons and links to enhance your online presence directly from your email signature.

Call-to-Action Buttons: Add interactive buttons to promote events, special deals, or direct recipients to specific web pages.

Email Tracking: Monitor email performance with tracking features that provide insights into opens and clicks.

Pricing:

Free Plan: Offers basic signature creation with limited templates and features.

Pro Plan: Starts at $4 per month, providing access to premium templates, advanced customization options, and email tracking capabilities.

Rating: ★★★★☆ (4.7/5)

#15. Consumer Directory: Email List Builder

Best for: Marketers and businesses looking to expand their email lists with verified consumer contact information.

What It Does:The Customers.ai Consumer Directory is a powerful tool for building high-quality email lists. It provides access to a vast database of verified consumer contact information, allowing businesses to target potential customers effectively. The directory is designed to integrate seamlessly with your existing marketing tools, streamlining the list-building process and enabling precise targeting for outreach campaigns.

Standout Features:

Extensive Consumer Database: Gain access to millions of verified consumer email addresses and contact details.

Advanced Filtering Options: Filter contacts by demographics, location, interests, and more to create highly targeted lists.

Real-Time Updates: Ensure your contact list is always fresh with up-to-date information.

Integration Ready: Easily sync with platforms like Klaviyo, HubSpot, and Mailchimp for smooth email campaign execution.

Compliance-First Approach: Operates with full compliance to email marketing regulations like GDPR and CAN-SPAM.

Pricing:

Custom pricing based on the size and scope of your desired email list.

Rating: ★★★★★ (4.9/5)

#16. Klaviyo: Email Marketing Automation 

Best for: Ecommerce businesses seeking advanced email and SMS marketing automation with robust segmentation capabilities.

What It Does:Klaviyo is a leading email marketing platform specifically designed for ecommerce brands. It helps businesses create personalized, data-driven email campaigns and automations. By integrating with ecommerce platforms like Shopify and WooCommerce, Klaviyo enables businesses to use customer data for hyper-targeted marketing efforts.

Standout Features:

Advanced Segmentation: Create targeted email campaigns based on purchase behavior, browsing history, and engagement levels.

Automated Flows: Set up sophisticated email sequences, such as welcome series, abandoned cart reminders, and post-purchase follow-ups.

Revenue Tracking: Monitor how much revenue each email campaign generates with built-in analytics.

Multi-Channel Marketing: Combine email and SMS marketing for a cohesive customer experience.

Pre-Built Templates: Use customizable templates to create professional emails without starting from scratch.

Pricing:

Free Plan: Available for up to 250 contacts with 500 email sends per month.

Paid Plans: Start at $20 per month, scaling based on the number of contacts and features.

Rating: ★★★★★ (4.8/5)

#17: Phantombuster: Best Email Scraper

Best for: Businesses and marketers seeking to automate data extraction and lead generation from various online platforms.

What It Does:Phantombuster is a versatile automation tool that enables users to extract data from websites and social media platforms without manual effort. It offers a range of “Phantoms”—pre-configured automation scripts—that can scrape emails, gather contact information, and perform other repetitive tasks, streamlining lead generation and data collection processes.

Standout Features:

Email Extractor: Automatically visits websites to extract all publicly available email addresses, exporting them into a CSV file for easy access.

Professional Email Finder: Finds professional email addresses using a person’s full name and company name, verifying their authenticity to ensure accuracy.

Data Scraping Crawler: Extracts emails, phone numbers, and social media information from any site on the web, providing comprehensive contact details.

Integration Capabilities: Connects seamlessly with CRM systems and other marketing tools to incorporate extracted data into existing workflows.

Cloud-Based Automation: Operates in the cloud, allowing users to run automations without keeping their browser open or computer on.

Pricing:

Free Plan: Offers limited usage with access to basic features.

Paid Plans: Start at $59 per month, providing additional usage credits and access to advanced functionalities.

Rating: ★★★★☆ (4.5/5)

#18: Sendlane: Email Marketing Automation

Best for: Ecommerce businesses seeking a unified platform for email and SMS marketing automation.

What It Does:Sendlane is a comprehensive marketing automation platform designed to help ecommerce brands create personalized email and SMS campaigns. It offers advanced automation features, real-time analytics, and deep-data integrations, enabling businesses to engage customers effectively and drive revenue growth.

Standout Features:

Behavior-Based Automation: Trigger personalized messages based on customer actions, such as purchases or site visits.

Multivariable Segmentation: Segment audiences using various data points to deliver targeted content.

Real-Time Analytics: Monitor campaign performance with up-to-the-minute data and insights.

Pre-Built Funnels: Access over 50 pre-built email and SMS funnels tailored for ecommerce, including abandoned cart recovery and post-purchase sequences.

24/7 Customer Support: Benefit from around-the-clock assistance with an average 30-second response time.

Pricing:

Growth Plan: Starts at $83 per month for up to 5,000 contacts, including all features and access to the eCommerce Academy.

Pro Plan: Begins at $640 per month for up to 50,000 contacts, offering additional benefits like custom onboarding and monthly program reviews.

Rating: ★★★★★ (4.7/5)

#19: Email Octopus: Email Marketing for SMB

Best for: Small to medium-sized businesses (SMBs) seeking a cost-effective and user-friendly email marketing solution.

What It Does:EmailOctopus is an email marketing platform designed to help SMBs create, send, and analyze email campaigns with ease. It offers a range of features tailored to meet the needs of growing businesses, including customizable templates, automation workflows, and detailed analytics. With its intuitive interface and affordable pricing, EmailOctopus enables businesses to engage their audience effectively without breaking the bank.

Standout Features:

Drag-and-Drop Email Builder: Create professional-looking emails effortlessly using a user-friendly editor.

Automation Workflows: Set up automated email sequences to nurture leads and engage customers based on their behavior.

Segmentation: Target specific groups within your audience to deliver personalized content that resonates.

Analytics and Reporting: Gain insights into campaign performance with metrics such as open rates, click-through rates, and conversions.

Integrations: Connect seamlessly with popular platforms like Shopify, WordPress, and Zapier to streamline your marketing efforts.

Pricing:

Free Plan: Available for up to 2,500 subscribers and 10,000 emails per month, including core features.

Pro Plan: Starts at $24 per month, offering additional benefits like priority support and advanced analytics.

Rating: ★★★★☆ (4.5/5)

#20: Mail Genius: Email Spam Tester 

Best for: Email marketers and businesses aiming to enhance email deliverability by identifying and resolving potential spam issues.

What It Does:MailGenius is a free email testing tool designed to help users assess whether their emails are likely to land in recipients’ inboxes or be flagged as spam. By analyzing various elements of your email, including content, authentication records, and server configurations, MailGenius provides actionable insights to improve deliverability rates.

Standout Features:

Spam Analysis: Identifies components within your email that may trigger spam filters, offering recommendations to address these issues.

Authentication Checks: Verifies SPF, DKIM, and DMARC records to ensure your emails are properly authenticated.

Blacklist Monitoring: Checks if your IP address or domain is listed on major blacklists, which could affect deliverability.

HTML Best Practices: Analyzes your email’s HTML structure to detect elements that may resemble spam.

Subject Line Optimization: Provides advice on optimizing your subject line’s length and structure to improve open rates.

Pricing:

Free: MailGenius offers its comprehensive email testing services at no cost, making it accessible for businesses of all sizes.

Rating: ★★★★★ (4.8/5)

#21: Warmy.io: Email Warmup Tool 

Best for: Businesses and individuals aiming to enhance email deliverability by establishing a positive sender reputation.

What It Does:Warmy.io is an email warmup tool designed to improve the deliverability of your emails by gradually building your sender reputation. It automates the process of sending and receiving emails, mimicking natural email interactions to signal to email service providers that your email address is trustworthy. This process helps prevent your emails from landing in spam folders and increases the likelihood of them reaching your recipients’ inboxes.

Standout Features:

Automated Warmup Process: Warmy.io handles the entire warmup process automatically, saving you time and effort.

AI-Powered Optimization: Utilizes artificial intelligence to adjust the volume and frequency of emails based on your domain and IP reputation, ensuring an effective warmup.

Deliverability Monitoring: Provides detailed metrics on your domain reputation, including health scores, to help you monitor and improve your email deliverability.

Customizable Warmup Schedules: Allows you to tailor the warmup process to your specific needs, accommodating different domains and IP addresses.

Integration Capabilities: Seamlessly integrates with various email service providers, making it easy to incorporate into your existing email marketing strategy.

Pricing:

Free Trial: Warmy.io offers a 7-day free trial to let you experience its features.

Paid Plans: Pricing details are available on their website, with plans designed to suit different business needs.

Rating: ★★★★☆ (4.7/5)

#22: Email Validator & Anti-Spam Tool

What It Does:

The Customers.ai Email Validator & Anti-Spam Tool is a must-have for marketers who want their emails to actually reach their audience. It works behind the scenes using AI to catch invalid, risky, or throwaway email addresses before they clog up your list. By keeping things clean, it helps protect your sender reputation and gives your emails a better chance of landing where they belong—the inbox. Plus, it flags anything in your email content that might scream “spam” to filters, so you can fix it before you hit send. Simple, smart, and exactly what your email campaigns need.

Standout Features:

Real-Time Email Validation: Instantly checks email addresses during sign-ups to keep your list squeaky clean.

Anti-Spam Content Scanner: Identifies spam trigger words, links, and formatting issues before sending.

Integration-Friendly: Easily connects with your favorite platforms like Klaviyo, HubSpot, and Shopify.

Detailed Reporting: Offers actionable insights to improve email deliverability and campaign performance.

Pricing: Flexible pricing options starting at $99/month, with a free trial available.

Rating: ★★★★★ (4.9/5)

#23: GlockApps: Email Deliverability Checker

Best for: Email marketers and businesses aiming to enhance their email deliverability and ensure messages reach recipients’ inboxes.

What It Does:GlockApps is a comprehensive email deliverability testing and monitoring platform. It allows users to assess where their emails land—be it the inbox, spam folder, or promotions tab—across various email service providers. By providing insights into potential deliverability issues, GlockApps helps users optimize their email campaigns for better performance.

Standout Features:

Inbox Placement Testing: Evaluate email placement across major ISPs to identify deliverability challenges.

Spam Filter Analysis: Analyze emails against common spam filters to detect elements that may trigger spam classifications.

Authentication Checks: Verify SPF, DKIM, and DMARC records to ensure proper email authentication.

Blacklist Monitoring: Monitor IP addresses and domains against over 50 industry blacklists, including Spamhaus and SORBS.

Content Analysis: Assess email content for potential issues that could affect deliverability, such as risky URLs or HTML errors.

Pricing:

Free Plan: Offers up to 2 free email tests.

Paid Plans: Start at $59 per month, providing additional tests and advanced features.

Rating: ★★★★☆ (4.8/5)

#24: Omnisend: Email Subject Line Checker 

Best for: Email marketers aiming to enhance open rates by optimizing subject lines through data-driven analysis.

What It Does:Omnisend’s Email Subject Line Tester is a free tool designed to evaluate and improve your email subject lines. By analyzing factors such as length, wording, and scannability, it provides actionable insights to help you craft subject lines that capture attention and encourage opens.

Standout Features:

Subject Line Scoring: Receive a percentage score indicating the effectiveness of your subject line.

Length Analysis: Assess character and word count to ensure optimal length for various devices and email clients.

Wording Evaluation: Identify positive and negative words, with suggestions for replacements to enhance engagement.

Scannability Check: Analyze capitalization and numerical usage to improve readability and appeal.

Best Practices Guidance: Access tips and recommendations based on industry standards to refine your subject lines.

Pricing: The Email Subject Line Tester is available for free, with no sign-up required.

Rating: ★★★★☆ (4.5/5)

#25: MailCharts: Email Competitive Intelligence  

Best for: Email marketers and businesses seeking to gain insights into competitors’ email strategies and industry trends.

What It Does:MailCharts is a comprehensive email marketing intelligence platform that provides access to a vast database of email campaigns from various brands and industries. It enables users to analyze competitors’ email strategies, discover industry trends, and gather inspiration for their own campaigns. By offering detailed analytics and examples, MailCharts helps businesses optimize their email marketing efforts and stay ahead in the competitive landscape.

Standout Features:

Extensive Email Database: Access thousands of email campaigns across multiple industries for analysis and inspiration.

Competitive Analysis: Compare your email strategies with competitors to identify strengths, weaknesses, and opportunities.

Industry Insights: Stay informed about the latest trends and best practices in email marketing within your sector.

Custom Lists: Create and share curated lists of email examples to collaborate with your team and streamline campaign planning.

Advanced Search Filters: Utilize sophisticated search options to find specific email types, designs, or strategies relevant to your needs.

Pricing:

Free Account: Provides limited access to features and email examples.

Paid Plans: Offer comprehensive access to the platform’s features and extensive email database. Pricing details are available on the MailCharts website.

Rating: ★★★★☆ (4.6/5)

Email Deliverability Hacks:

Unlocking Inboxes

HOSTED BY

Larry Kim

Founder and CEO, Customers.ai

Free Webinar: Watch Now

How Email Marketing Tools Evolve With Your Business

Your email marketing needs as a scrappy startup won’t be the same as when you’re managing a six-figure email list. That’s where scalability comes in. A great email marketing tool grow with you, adapting to new challenges as your business evolves.

Here are six things to look for:

1. Flexible Pricing Plans

When you’re starting out, free or low-cost plans are a lifesaver. But as your list grows, you’ll need a tool that offers tiered pricing so you’re only paying for what you need. Look for tools that make it easy to upgrade without hidden fees or unnecessary features.

2. Advanced Automation for Growing Campaigns

A welcome email might be enough in the early days but when you’re managing multiple product lines or audiences, you’ll need robust automation. Think tools that can handle advanced workflows, like multi-step campaigns triggered by specific customer actions.

3. Better Segmentation as Your Audience Expands

A small audience might not need much segmentation but when your list hits thousands (or more), you’ll need tools that let you target specific groups with precision.

Look for dynamic segmentation that updates in real-time as customers interact with your brand.

4. Analytics That Scale Up

Basic open and click rates are fine for a beginner but scaling businesses need deeper insights. Tools that offer heatmaps, revenue tracking, and customer lifecycle metrics help you understand what’s working and where to optimize.

5. Integrations That Grow With You

When you’re just starting out, basic integrations might be enough. But as you scale, you’ll need tools that can connect with CRMs, ecommerce platforms, and analytics tools seamlessly. Make sure your email platform can keep up.

6. Support for Larger Campaigns

As your list grows, so do your campaigns. Look for tools that can handle high-volume sending without deliverability issues and that offer priority support for troubleshooting when things get complex.

Whether you’re just starting out or hitting new growth milestones, scalable tools make it easier to connect with your audience and keep the momentum going.

As you think about your tool set, it’s important to remember that email marketing is a big part of your business’s growth. The right tools don’t just need to meet your current needs, they also need to adapt and scale as your business evolves.

So wether you’re looking for automation, segmentation, or powerful analytics, investing in the right platform can transform your campaigns from good to game-changing.

Ready to take your email marketing to the next level? Explore these tools, experiment, and find the perfect fit to grow with your business. Your inbox—and your bottom line—will thank you.

See Who Is On Your Site Right Now!

Get names, emails, phone numbers & more.

Try it Free, No Credit Card Required

Start Your Free Trial

Looking for More Marketing Tools?

If you need even more marketing tools, look no further. Browse through hundreds of more awesome tools in the following lists:

Free marketing software

Tools for digital marketing agencies

Marketing automation tools

Trending digital marketing tools

Social media marketing tools

Ecommerce marketing tools

Instagram marketing tools

B2B marketing tools

Viral marketing tools

Marketing tools for small business

Growth hacking tools

Conversion rate optimization (CRO) Tools

Marketing automation tools for agencies

Cross-channel marketing tools

Free advertising sites

Customer support chat tools

Affiliate marketing tools

Important Next Steps

See what targeted outbound marketing is all about. Capture and engage your first 500 website visitor leads with Customers.ai X-Ray website visitor identification for free.

Talk and learn about sales outreach automation with other growth enthusiasts. Join Customers.ai Island, our Facebook group of 40K marketers and entrepreneurs who are ready to support you.

Advance your marketing performance with Sales Outreach School, a free tutorial and training area for sales pros and marketers.

Email Marketing Tool FAQs

Q. What are email marketing tools?

Email marketing tools are software designed to create, send, track, and optimize email campaigns. They help businesses reach out to their customers and prospects through email, offering features like templates, automation, segmentation, and analytics to improve the effectiveness of email communications.

Q. Why use email marketing tools?

Email marketing tools are used to efficiently manage large-scale email campaigns, personalize messages for different audience segments, automate repetitive tasks, and analyze the success of email strategies. They save time, increase ROI, and enhance the ability to engage and convert recipients.

Q. How do email marketing tools improve ROI?

Email marketing tools improve ROI by allowing marketers to target specific segments of their audience with personalized messages, automate follow-up emails, and track the performance of campaigns. This targeted approach leads to higher engagement rates and conversions.

Q. Can email marketing tools help with GDPR compliance?

Yes, many email marketing tools include features to help businesses comply with GDPR, such as consent forms, easy unsubscribe options, and data management capabilities, ensuring that email campaigns respect privacy laws and regulations.

Q. What features should I look for in an email marketing tool?

Look for features like automation, segmentation, A/B testing, analytics, and integrations with other tools. These features help create more effective and personalized email campaigns, track their success, and streamline marketing workflows.

Q. How does automation in email marketing tools work?

Automation in email marketing tools allows you to set up triggers based on user actions or specified conditions, which then automatically send relevant emails to subscribers. This can include welcome emails, birthday messages, or follow-up emails after a purchase.

Q. How do email marketing tools help with A/B testing?

Email marketing tools facilitate A/B testing by allowing you to send two variations of an email to a small segment of your audience. You can test elements like subject lines, content, or send times to see which performs better and apply those insights to future campaigns.

Q. Can email marketing tools integrate with CRM software?

Yes, many email marketing tools can integrate with CRM software, allowing for seamless data exchange between the two systems. This helps businesses maintain updated customer information, personalize emails, and track interactions across both platforms.

Q. Are there free email marketing tools available?

Yes, there are several free email marketing tools available that offer basic features suitable for small businesses or those just starting out. However, as your business grows, you may need to upgrade to paid plans to access advanced features.

Q. How can email marketing tools track campaign success?

Tools track success through metrics like open rates, click-through rates, conversion rates, and bounce rates. This data helps marketers understand audience behavior and refine strategies for better performance.

Q. What is the difference between email marketing and automation tools?

Email marketing tools focus on creating and sending emails, while automation tools offer more advanced features like triggering emails based on user behavior, creating workflows, and integrating with other marketing tools for a seamless campaign across channels.

Q. How do I choose the best email marketing tool for my business?

Consider your business size, budget, marketing goals, and the specific features you need, such as automation, analytics, and integrations. Also, look for tools with a user-friendly interface and reliable customer support.

Q. Can email marketing tools assist with lead generation?

Yes, by creating engaging content and targeted campaigns, email marketing tools can help capture and nurture leads, moving them through the sales funnel towards conversion.

Q. What role do templates play in email marketing tools?

Templates help create visually appealing and consistent emails quickly and efficiently, without needing extensive design skills. They can be customized to fit your brand and message.

Q. How do email marketing tools manage subscriber lists?

These tools allow you to import, organize, and manage subscriber lists, including adding new subscribers, segmenting lists for targeted campaigns, and cleaning out inactive or unsubscribed contacts.

Q. What are the benefits of using AI in email marketing?

AI can help personalize emails at scale, predict the best send times, optimize email content for engagement, and analyze data to provide insights into improving campaign strategies.

Q. How does social media integration enhance email marketing?

Integrating social media allows you to extend the reach of your email content, encourage social sharing, and integrate your marketing efforts across channels for a unified brand message.

Q. What security features should email marketing tools have?

Look for features like secure data storage, GDPR compliance, reliable unsubscribe options, and protection against data breaches to ensure your and your subscribers’ data remains safe.

Q. How do email marketing tools track campaign success?

Email marketing tools track campaign success through analytics and reporting features. They measure key performance indicators like open rates, click-through rates, conversion rates, and bounce rates, providing insights into how well your emails are performing and where improvements can be made.

Q. What is the best time to send email campaigns?

The best time to send email campaigns varies depending on your audience’s habits, but data from email marketing tools can help identify optimal send times. Generally, mid-week mornings are considered effective for business audiences, while weekends may work better for consumer-focused emails.

Q. How can email marketing tools prevent spam complaints?

Email marketing tools prevent spam complaints by ensuring that emails are sent only to subscribers who have opted in, providing clear unsubscribe options, and following best practices for email content and frequency. They also help maintain a healthy sender reputation to reduce the risk of being flagged as spam.

Q. What role does personalization play in email marketing tools?

Personalization plays a crucial role in increasing engagement and conversions. Email marketing tools offer personalization features that allow you to address subscribers by name, recommend products based on past purchases, and send targeted content based on user behavior and preferences.

Q. Can email marketing tools help in list building and management?

Yes, email marketing tools provide features for list building and management, such as signup forms, landing pages, and integration with social media platforms. They also offer capabilities for cleaning and segmenting lists to maintain high-quality, engaged subscriber bases.

Q. How do email marketing tools support mobile optimization?

Email marketing tools support mobile optimization by offering responsive design templates that automatically adjust to the screen size of smartphones and tablets. This ensures that your emails look great and are easy to read on any device.

Q. What is dynamic content in email marketing?

Dynamic content refers to the ability of email marketing tools to change parts of an email’s content based on the recipient’s data or behavior. This can include product recommendations, content preferences, or personalized offers, making each email more relevant to the individual subscriber.

Q. How does social media integration work with email marketing tools?

Social media integration allows you to connect your email marketing campaigns with your social media accounts, enabling you to share your emails on social platforms and add social follow buttons to your emails. This helps broaden your reach and engage with your audience across multiple channels.

Q. What is the significance of deliverability in email marketing?

Deliverability is critical in email marketing as it measures the ability of your emails to reach the subscriber’s inbox without being blocked or marked as spam. Good deliverability depends on a healthy sender reputation, adherence to email standards, and engaging content.

Q. How can I measure the ROI of email marketing campaigns?

You can measure the ROI of email marketing campaigns by tracking metrics such as conversion rates, revenue generated from email campaigns, and overall engagement. Email marketing tools provide detailed reports that help calculate ROI by comparing the revenue against the cost of running the campaigns.
The post Email Marketing Tools: 25 Powerful Platforms to Level Up Engagement in 2025 appeared first on Customers.ai.