Skip to content
CostReveal
Menu

Connecting Datadog to CostReveal, setup guide

The CostReveal team, , 7 minute read

Observability spend grows quietly: a new service sends more logs, a team adds hosts, and the Datadog invoice moves before anyone notices. This guide covers how CostReveal reads what Datadog states your organizations cost, product by product, the difference between Datadog's estimated and final costs, each setup step, how cost attribution by tag explains spend, how organizations become accounts, and how closed months are checked against Datadog's own total.

What the Datadog connection reads

CostReveal reads what Datadog states your parent organization, and every organization under it, cost, product by product. It reads three things:

  • Final costs, once Datadog closes a month, by the 16th of the next: each organization's cost by product, such as infrastructure hosts, APM hosts and indexed logs, and by charge type, committed or on demand. These are invoiced figures.
  • Estimated costs, for the current and previous month until each closes: each day's cost, stated up to three days late. They are shown as not yet invoiced.
  • Cost attribution by tag, where your Datadog plan has it: a closed month's cost by the values of the tags your organization attributes cost by.

A product's committed and on demand charges are counted once: Datadog also states a total beside them, and that total is never added to them.

What you need

  • Access to your Datadog parent organization's settings, to create keys.
  • Someone who holds both the usage_read and billing_read permissions, to create the application key.
  • An owner or admin role in CostReveal.

Step by step setup

1. Start the connection

In CostReveal, go to Connections and choose Connect next to Datadog.

2. Create the keys

In the parent organization, under Organization Settings, create:

  • An API key.
  • An application key, scoped to usage_read and billing_read, created by someone who holds both permissions. The scope limits the key to reading usage and billing.

3. Choose your site and verify

Choose your Datadog site, such as datadoghq.com or datadoghq.eu, enter both keys, and choose Verify and connect. Only the format is checked as you type. When you verify, CostReveal reads this month's costs with the keys before saving anything. If something is wrong, it says which: an API key Datadog does not know on that site, or keys that cannot read costs.

The keys are stored encrypted and never shown again. Revoking either in Datadog ends access at once.

Freshness and history

Datadog estimates each day's cost up to 72 hours later. CostReveal reads every six hours, and reads the current and previous month each time; the first read also brings in the twelve months before the current one, as far as Datadog holds them. Where Datadog holds no older months, Connections says how far back it reached, and each read asks again. A month is read again only when what Datadog states for it has changed. When Datadog closes a month, its final costs replace the estimates.

Connection states

Under Connections, the connection is always in one of four states:

  • Setup not finished: the keys have not been verified yet.
  • Waiting for first data: the keys were verified, and the first read is due.
  • Connected: costs were read and the latest read succeeded.
  • Needs attention: the latest read failed, for example because a key was revoked. Everything already read is kept.

How Datadog spend is explained

  • Direct: the service, the Datadog product, which Datadog states itself.
  • Inferred: the team, feature or customer, from Datadog's cost attribution by tag, read with the project's tag keys. If your organization attributes cost by a team tag, a closed month's cost arrives split by team value, and that spend is inferred from it.
  • Allocated: a share of a cost an allocation rule divides, by fixed percentages, evenly, or in proportion to what each owner already explains.
  • Unknown: spend no attribution or rule explains. It is shown as unknown, never spread across teams.

What cost attribution by tag covers

Cost attribution by tag is a Datadog feature, and depends on your plan. It uses the tags your organization configures for cost attribution in Datadog, and it covers closed months only. What it does not cover stays one line per product, so each month still adds up exactly to its final cost. The current month, which is estimated, is shown by product until it closes.

Organizations as accounts

Each organization under the parent is its own account in CostReveal. So where a whole organization belongs to one team, an allocation rule on the account names its owner, and where several share one, a rule can split it.

How figures are checked

The ledger against attribution. Every amount is recorded as Datadog states it. Estimates are replaced by final costs when a month closes, recorded as corrections that keep what was said before, and attribution runs again. Attribution on every dimension must add up exactly to the ledger.

The ledger against Datadog's own total. For closed months, each organization's final costs are compared with Datadog's own total cost. A difference is flagged as a variance, never adjusted away. Open months are estimates, and Home says so, as not yet invoiced.

Getting more of Datadog spend explained

  1. Turn on cost attribution by tag in Datadog, where your plan has it, for a tag key the project reads, such as team.
  2. Write a rule per organization that belongs to one team.
  3. Split shared organizations with a rule, so shared monitoring is allocated in the open.
  4. Watch the estimated month in Explore by product, to see a rise before the invoice.

Troubleshooting

  • Verify says Datadog does not know the API key. Check you chose the site the organization is on, such as datadoghq.eu rather than datadoghq.com.
  • Verify says the keys cannot read costs. Check the application key is scoped to usage_read and billing_read, and was created by someone who holds both.
  • No split by tag. Cost attribution by tag depends on your Datadog plan and covers closed months only.
  • The current month changes after the fact. Estimates arrive up to three days late, and are replaced by final costs when Datadog closes the month.

Using Datadog figures in CostReveal

Once Datadog spend is read, it appears everywhere the rest of your spend does:

  • Explore groups it by provider, service, account, region, team, feature, customer or any custom dimension you define, and can filter to any of them. Every row shows how much of it is direct, inferred, allocated or unknown.
  • Budgets can be set on Datadog as a provider, or on any one of its services, with alerts when spend nears or passes the limit, or when the month is forecast past it.
  • Insights forecasts the month once it has three complete days, with a range and a confidence, and flags a day when a service spent outside its normal range: the median of the same weekday over the four weeks before.
  • Reports export the figures as CSV or a PDF, once or on a weekly or monthly schedule.
  • Estimated and final, side by side. The open month is Datadog's estimate, shown as not yet invoiced, and a forecast of the month is built from those estimates. When Datadog closes the month, final costs replace the estimates, and the month is checked against Datadog's own total.

Security and access

  • What CostReveal holds. The API key and the application key, scoped to usage_read and billing_read. It is encrypted with AES-256-GCM before it is stored, under keys that can be rotated, and it is never shown again after you enter it.
  • What it is used for. Reading only. CostReveal never changes anything in your account.
  • Who can change the connection. Only owners and admins in CostReveal can connect, change or remove a connection, and each of those actions is recorded in the organization's audit log, with who did it and when.
  • Isolation. Every figure is stored in CostReveal's database with your organization isolated by PostgreSQL row level security, enforced in the database itself, below the application.
  • Ending access. Revoke either key in Datadog. CostReveal keeps every figure already read and says that new data has stopped.

Removing the connection

Removing the connection under Connections deletes every figure it brought in from the project. Revoke the keys in Datadog too. To stop reading without removing anything, revoke either key: CostReveal keeps every figure already read and tells you new data has stopped.

Frequently asked questions

Which keys does CostReveal need? An API key, and an application key scoped to usage_read and billing_read, both from the parent organization.

Why do the current month's figures change? They are Datadog's estimates, stated up to three days late, and replaced by final costs when Datadog closes the month, by the 16th of the next.

Is my Datadog spend checked against anything? Yes, for closed months: each organization's final costs are compared with Datadog's own total cost.

How do I split Datadog spend by team? Turn on cost attribution by tag in Datadog for a key the project reads, and write rules for organizations that belong to one team.

Are committed and on demand charges double counted? No. Each product's committed and on demand charges are counted once, and the total Datadog states beside them is never added.

How much history is brought in? The first read brings in the current month and the twelve before it, as far as Datadog holds them. Where Datadog holds no older months, Connections says how far back it reached, and each read asks again. After that, each read covers the current and previous month.

Keep reading

More setup guides