{"id":250,"date":"2026-02-24T00:00:00","date_gmt":"2026-02-24T00:00:00","guid":{"rendered":"https:\/\/wordpress.securinsight.ca\/index.php\/2026\/02\/24\/pipelines-workers-dropped-event-metrics-typed-pipelines-bindings-and-improved-setup\/"},"modified":"2026-02-24T00:00:00","modified_gmt":"2026-02-24T00:00:00","slug":"pipelines-workers-dropped-event-metrics-typed-pipelines-bindings-and-improved-setup","status":"publish","type":"post","link":"https:\/\/wordpress.securinsight.ca\/index.php\/2026\/02\/24\/pipelines-workers-dropped-event-metrics-typed-pipelines-bindings-and-improved-setup\/","title":{"rendered":"Pipelines, Workers – Dropped event metrics, typed Pipelines bindings, and improved setup"},"content":{"rendered":"

Cloudflare Pipelines<\/a> ingests streaming data via Workers<\/a> or HTTP endpoints, transforms it with SQL, and writes it to R2<\/a> as Apache Iceberg tables. Today we’re shipping three improvements to help you understand why streaming events get dropped, catch data quality issues early, and set up Pipelines faster.<\/p>\n

Dropped event metrics<\/h4>\n

When stream<\/a> events don’t match the expected schema, Pipelines accepts them during ingestion but drops them when attempting to deliver them to the sink<\/a>. To help you identify the root cause of these issues, we are introducing a new dashboard and metrics that surface dropped events with detailed error messages.<\/p>\n

\"The<\/p>\n

Dropped events can also be queried programmatically via the new pipelinesUserErrorsAdaptiveGroups<\/code> GraphQL dataset. The dataset breaks down failures by specific error type (missing_field<\/code>, type_mismatch<\/code>, parse_failure<\/code>, or null_value<\/code>) so you can trace issues back to the source.<\/p>\n

\n
\n
query<\/span> <\/span>GetPipelineUserErrors<\/span>(<\/span><\/div><\/div>
  <\/span>$accountTag<\/span>:<\/span> <\/span>String<\/span>!<\/span><\/div><\/div>
  <\/span>$pipelineId<\/span>:<\/span> <\/span>String<\/span>!<\/span><\/div><\/div>
  <\/span>$datetimeStart<\/span>:<\/span> <\/span>Time<\/span>!<\/span><\/div><\/div>
  <\/span>$datetimeEnd<\/span>:<\/span> <\/span>Time<\/span>!<\/span><\/div><\/div>
) <\/span>{<\/span><\/div><\/div>
  <\/span><\/span>viewer <\/span>{<\/span><\/div><\/div>
    <\/span><\/span>accounts(<\/span>filter<\/span>:<\/span> { <\/span>accountTag<\/span>:<\/span> $accountTag }) <\/span>{<\/span><\/div><\/div>
      <\/span><\/span>pipelinesUserErrorsAdaptiveGroups(<\/span><\/div><\/div>
        <\/span>limit<\/span>:<\/span> <\/span>100<\/span><\/div><\/div>
        <\/span>filter<\/span>:<\/span> {<\/span><\/div><\/div>
          <\/span>pipelineId<\/span>:<\/span> $pipelineId<\/span><\/div><\/div>
          <\/span>datetime_geq<\/span>:<\/span> $datetimeStart<\/span><\/div><\/div>
          <\/span>datetime_leq<\/span>:<\/span> $datetimeEnd<\/span><\/div><\/div>
        <\/span><\/span>}<\/span><\/div><\/div>
        <\/span>orderBy<\/span>:<\/span> [<\/span>count_DESC<\/span>]<\/span><\/div><\/div>
      <\/span><\/span>) <\/span>{<\/span><\/div><\/div>
        <\/span><\/span>count<\/span><\/div><\/div>
        <\/span><\/span>dimensions <\/span>{<\/span><\/div><\/div>
          <\/span><\/span>errorFamily<\/span><\/div><\/div>
          <\/span><\/span>errorType<\/span><\/div><\/div>
        <\/span>}<\/span><\/div><\/div>
      <\/span>}<\/span><\/div><\/div>
    <\/span>}<\/span><\/div><\/div>
  <\/span>}<\/span><\/div><\/div>
}<\/span><\/div><\/div><\/code><\/pre>\n
\n
<\/div>\n<\/div>\n<\/figure>\n<\/div>\n
For the full list of dimensions, error types, and additional query examples, refer to User error metrics<\/a>.<\/p>\n
Typed Pipelines bindings<\/h4>\n
Sending data to a Pipeline from a Worker previously used a generic Pipeline<PipelineRecord><\/code> type, which meant schema mismatches (wrong field names, incorrect types) were only caught at runtime as dropped events.<\/p>\n
Running wrangler types<\/code> now generates schema-specific TypeScript types for your Pipeline bindings<\/a>. TypeScript catches missing required fields and incorrect field types at compile time, before your code is deployed.<\/p>\n
\n
\n
declare<\/span> <\/span>namespace<\/span> <\/span>Cloudflare<\/span> <\/span>{<\/span><\/div><\/div>
  <\/span>type<\/span> <\/span>EcommerceStreamRecord<\/span> <\/span>=<\/span> <\/span>{<\/span><\/div><\/div>
    <\/span><\/span>user_id<\/span>:<\/span> <\/span>string<\/span>;<\/span><\/div><\/div>
    <\/span><\/span>event_type<\/span>:<\/span> <\/span>string<\/span>;<\/span><\/div><\/div>
    <\/span><\/span>product_id<\/span>?:<\/span> <\/span>string<\/span>;<\/span><\/div><\/div>
    <\/span><\/span>amount<\/span>?:<\/span> <\/span>number<\/span>;<\/span><\/div><\/div>
  <\/span>};<\/span><\/div><\/div>
  <\/span>interface<\/span> <\/span>Env<\/span> <\/span>{<\/span><\/div><\/div>
    <\/span><\/span>STREAM<\/span>:<\/span> <\/span>import<\/span>(<\/span>\"cloudflare:pipelines\"<\/span>).<\/span>Pipeline<\/span><<\/span>Cloudflare<\/span>.<\/span>EcommerceStreamRecord<\/span>>;<\/span><\/div><\/div>
  <\/span>}<\/span><\/div><\/div>
}<\/span><\/div><\/div><\/code><\/pre>\n
\n
<\/div>\n<\/div>\n<\/figure>\n<\/div>\n
For more information, refer to Typed Pipeline bindings<\/a>.<\/p>\n
Improved Pipelines setup<\/h4>\n
Setting up a new Pipeline previously required multiple manual steps: creating an R2 bucket, enabling R2 Data Catalog, generating an API token, and configuring format, compression, and rolling policies individually.<\/p>\n
The wrangler pipelines setup<\/code> command now offers a Simple<\/strong> setup mode that applies recommended defaults and automatically creates the R2 bucket<\/a> and enables R2 Data Catalog<\/a> if they do not already exist. Validation errors during setup prompt you to retry inline rather than restarting the entire process.<\/p>\n
For a full walkthrough, refer to the Getting started guide<\/a>.<\/p>","protected":false},"excerpt":{"rendered":"
Cloudflare Pipelines ingests streaming data via Workers or HTTP endpoints, transforms it with SQL, and writes it to R2 as Apache Iceberg tables. Today we’re shipping three improvements to help you understand why streaming events get dropped, catch data quality issues early, and set up Pipelines faster. Dropped event metrics When stream events don’t match […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-250","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/posts\/250","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/comments?post=250"}],"version-history":[{"count":0,"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/posts\/250\/revisions"}],"wp:attachment":[{"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/media?parent=250"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/categories?post=250"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wordpress.securinsight.ca\/index.php\/wp-json\/wp\/v2\/tags?post=250"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}