Ramp's Real-Time Data Operations | Materialize

Summary:

Ramp’s Data Platform team builds key operational data capabilities like account takeover detection on Materialize primarily for three reasons:

1. Timeliness - Moving from 1hr+ of latency in a batch model to 1-3 second latency in Materialize means stopping fraudsters before they can act, reducing losses and improving margins for Ramp, and creating a safer experience for customers.
2. Cost - By moving operational workloads to Materialize, Ramp could turn off a high-cost category of workload on their analytical data warehouse.
3. Consistency - Internal teams can operate on the same consistent and quality-controlled data, without compromising on freshness of results.

About Ramp

Ramp is the finance automation platform uniquely designed to save businesses their most valuable resources—time and money. By integrating corporate cards with refreshingly modern software to manage all non-payroll spend, Ramp turns typically painful finance processes into an effortless and delightful experience.

The Data Platform team at Ramp goes well beyond managing pipelines for BI tools. They actively partner with product, engineering and design to help Ramp ship products that translate the wealth of data in their platform into value for their customers.

Operational capabilities, where data is used for more than just analysis, were a hard requirement for delivering on this expanded mandate of the data team. Ryan Delgado, Staff Software Engineer on the team, looked to Materialize to add those capabilities with minimal disruption to the SQL and dbt workflows that had worked so well for the company so far.

Stopping fraud before it affects customers

The team’s first workload on Materialize focused on detecting when fraudsters attempt to take over customer accounts. Account Takeovers (ATOs) can result in poor customer experience, reputational harm for Ramp, and of course significant monetary losses. ATO events happen regularly, and usually span over several weeks. The typical lifecycle of an ATO event is:

• Fraudsters host their own identical replica of Ramp’s login page.
• They then purchase Google Ads and serve the fake login page in these ads.
• Unsuspecting Ramp customers arrive at the fake login page via the ads. They “log in” to Ramp, but instead of logging in to Ramp they’ve been phished.
• With the users credentials in hand, the fraudsters login, generate new Ramp Virtual Cards, and buy easily-fenced items like power tools.

The team needed a way to stop account takeover and other types of abuse before it impacted their customers.

Initial prototype in an analytical warehouse

Ramp was able to quickly prototype a rules-based fraud detection approach using SQL transformations in their analytical data warehouse. The general approach was to build a sort of “feature table” that computed key input metrics for accounts and transactions. The table is used by a service that layers in more logic to ultimately make the decision of whether to flag a transaction.

Prototyping on the analytics warehouse in SQL allowed the Data Platforms team to work closely with the Risk Ops team to own the logic and iterate, but the end-to-end latency of the system started out at 30 minutes and continued to grow to over an hour as their company scaled.

Before: Ramp's Analytical Architecture not well suited to operational work.

They needed a way to take batch business logic and “ship it” as a service running continuously. Materialize provided that, in an event-driven model with dramatically lower end-to-end data latency.

Going Operational in Materialize

Like with their analytics warehouse, most of the initial build-out of Materialize at Ramp was focused on establishing secure and performant pipelines from production data sources.

Ramp established streaming replication from their primary database to Materialize via the fully-managed Postgres Source, which watches the write-ahead log (WAL) on Postgres for updates and immediately syncs them to Materialize.

After: Ramp's Operational Architecture with Materialize

SQL Build-Out

Once data was flowing, the team was able to port the rules-based fraud detection SQL logic to Materialize with minimal changes. This was critical. Other approaches they looked at, like stream processors, would have required a complete rewrite of logic at a significant cost in data engineering time, and future updates would continue to diverge from the familiar SQL in the analytical warehouse.

With Materialize, not only could they keep the logic consistent in SQL, by using the Materialize dbt adapter, the team could keep it all organized and version-controlled in a dbt project backed by a git repo.

Results

With the first operational workload active in production on Materialize, the team at Ramp saw positive results in several areas:

Reduced analytical warehouse cost: While the analytical data warehouse proved a good place for prototyping, getting ahead of the bad actors was not possible within the constraints of batch systems, and approaching those limits resulted in high compute costs. With the alerts running in Materialize, Ramp could turn off the workflows in their analytical warehouse to reduce costs.

60% Reduction in losses from ATO: Thanks to the alert views running on Materialize, multiple rounds of coordinated account take-over attacks have already been intercepted. The Ramp team is able to intervene, block the bad actors from the accounts, and lock fraudulently-created cards before they can be used to make illegitimate purchases, saving Ramp hundreds of thousands of dollars per year and creating a more secure and stable experience for Ramp customers.

Ability to continue to move quickly: Lastly, the familiarity of the SQL control layer in Materialize means the Ramp team can iterate quickly on alert logic to stay ahead of fraudsters, and the strong consistency of results means Ramp can operate on alerts with confidence.