What Are Healthcare Data Standards, and Why Doesn’t Payer Data Arrive Ready to Use?

Many interoperability roadmaps assume that because healthcare data “standards” exist, external data arrives ready to use.

For claims and eligibility data, the assumption rarely holds. The gap shows up in places executives notice. A new payer feed takes months instead of weeks. Settlement numbers do not tie to the payer’s report, and teams struggle to explain why.

Moving data and making it usable are two different jobs. At a recent Becker’s Healthcare IT panel, the moderator asked speakers to separate the successful transfer of healthcare data from the work of making that data usable and trusted once it arrives.

One panelist compared incoming data to an IKEA box that holds every part but has no labels and no instructions. The delivery worked, yet the receiving team still must figure out what each piece is and how the pieces fit together. 

“We have gotten very good at moving the box, and packing it, and moving it. The challenge here is opening it, and understanding it, and assembling it.” – Panelist, Becker’s Healthcare panel

She added that as AI use grows, governed data matters even more. Her point was about context: the organization that receives the data may read it differently from the organization that sent it, and closing that gap takes real work after the transfer is complete.

In HDI’s 2025 Market Report, 45% of surveyed organizations said integrating data from a new payer took five months or more.

Do healthcare data standards make external data plug and play?

No. Not on their own. Healthcare data standards define a shared vocabulary for transactions and codes. They do not guarantee that two organizations use the vocabulary the same way, and much of the claims data health systems use for analytics does not travel inside a standard transaction.

It helps to know where healthcare data standards end and where payer-specific practices begin. Interoperability moves data between organizations, but it does not make that data usable or trusted once it arrives.

Which standards govern claims and eligibility data?

Two standards families define the HIPAA transactions for claims and eligibility data. X12 governs medical claims, remittance, enrollment, and eligibility transactions, and NCPDP governs pharmacy claims. FHIR, an HL7 standard for API-based exchange, increasingly carries claims data as well. Code sets such as ICD-10-CM/PCS, CPT and HCPCS, and NDC sit underneath them. Even codes from these sets, the most standardized layer, need normalization before teams can match them. An NDC may arrive in its 10-digit form or in the 11-digit form that HIPAA billing transactions use, which pads one segment with a leading zero. ICD-10 codes may arrive with or without the decimal.

The FDA has also adopted a uniform 12-digit NDC format, effective March 7, 2033. The rule does not itself change the 11-digit format that HIPAA billing transactions use, but payers and health systems will need systems that handle both formats during the transition. 

What does each standard cover, and what does it not guarantee?

Standard	What it covers	What it does not guarantee
X12 837P, 837I, and 837D	Professional, institutional, and dental claims that providers submit to payers for payment	That a payer will send claims back in this format for analytics
X12 837 post-adjudicated claims data reporting	Adjudicated claims that payers and other reporting entities send to all-payer claims databases and other recipients	That a given payer uses it for the analytics feeds it sends to providers
X12 835	Claim payment and remittance advice from payer to provider	Visibility into care the provider did not deliver
X12 834	Benefit enrollment and maintenance	Consistent use of relationship codes and coverage dates across senders
X12 270/271	Eligibility inquiry and response for specific members, in real time or in batch	A population roster
NCPDP	Pharmacy claims at the point of sale and reporting of adjudicated pharmacy claims	Consistent downstream extracts or reversal handling
FHIR	API-based exchange of clinical data and, increasingly, claims data	Consistent implementation choices or complete financial detail

 

X12 also publishes separate implementation guides for post-adjudicated claims data reporting, which states such as New York require for all-payer claims database submissions, alongside the NCPDP Post Adjudication Standard for pharmacy claims. 

FHIR is also where federal policy is heading. Under the CMS Interoperability and Prior Authorization Final Rule (CMS-0057-F), Medicare Advantage organizations, Medicaid and CHIP fee-for-service programs and managed care plans, and qualified health plan issuers on the Federally Facilitated Exchanges must offer a Provider Access API by January 1, 2027, or by the rating period or plan year that begins on or after that date.

Payers must share claims and encounter data (without provider remittances or enrollee cost-sharing), USCDI clinical data, and certain prior authorization information. They share it only for patients they attribute to in-network or enrolled providers, and patients can opt out. The rule does not apply to employer-sponsored or self-funded plans, or to commercial plans sold outside the Federally Facilitated Exchanges. 

Where do healthcare data standards break down in practice?

Standards break down once payer data leaves the standard transaction. The claims and eligibility data health systems rely on for risk contracts and total cost of care usually arrives outside those standards, and it breaks in predictable ways.

A business intelligence analyst at a large Northeast health system spent six months trying to manage payer files internally, a path HDI’s white paper Managing Payer Data Internally examines in detail. The analyst described what arrived this way:

“The data is so different… comes in different files, different fields, values, everything is different from payer to payer.” – Business Intelligence Analyst, Large Health System, Northeast

Proprietary flat-file extracts. In HDI’s experience, most payers deliver claims data as their own extract, not as 837s. Each one sets its own column order, field names, delimiters, header and trailer records, and claim-level or line-level structure. HDI’s internal analysis shows that health systems with value-based care programs each manage an average of 28 active layouts from seven vendors, and some manage more than 75, each with its own shape. Some health systems work with vendors to manage to manage healthcare data.

“What you get from [one payer] is very different than what you get from [another payer].” – Manager, Data Services and Engineering, Large Health System, Southeast

Payers populate situational and optional fields differently. X12 implementation guides mark many elements as situational, required only under stated conditions, and senders apply those conditions differently. Extracts have no guide at all. One source puts the allowed amount in the paid field. One payer may send a reversal as negative lines, and another may send a full replacement claim.

Jonathan Kaye, HDI’s Founder and Executive Chairman, has worked with claims data since 1990. He has seen clients send HDI data in HL7 and FHIR formats, then explain that they don’t follow the standard because they find it more useful to put the deductible in a different field. He sums up the gap between specification and practice this way:

“Even within the context of standards, there are no standards.” – Jonathan Kaye, Founder and Executive Chairman, HDI

Layouts change over time. Payers update their own systems, and their file layouts can change with them. A new column appears, or a familiar field takes on a new meaning. The file still loads, so nothing fails visibly. In HDI’s 2025 Market Report, 89% of surveyed leaders identified keeping up with frequently changing payer file layouts as a claims data integration challenge. 

“Payers will change fields. Sometimes the layouts will change and we’re not even told.” – Director of Analytics, Large Health System, Southeast

Your team or your partner must catch each change and remap it before the data is safe to use. When a change slips through, the cost shows up later as broken trends and settlement figures that do not tie. Payers calculate performance from claims, so a provider whose claims data diverges from the payer’s may find it harder to understand and reconcile performance results with the payer. For more on why a file that loads still needs validation, see Integrated Isn’t Validated. 

How can teams bridge the specification and the file that actually arrives?

Teams bridge the gap with source-specific knowledge and validation each time a file arrives. The gap between what a standard or layout specification describes and what a payer actually sends is where External Data Management does its work.

Before committing an interoperability or analytics investment to a new external feed, health IT leaders can ask five questions:

1. Which standard and implementation guide version does this source actually follow, if any?
2. Which fields are optional or situational, and how does this source populate them?
3. How will we know when the layout or field usage changes?
4. How will we link reversals and corrected claims back to the original claim?
5. Will the output tie to the payer’s own settlement and performance reports?

For more on evaluating a claims data partner, see How to Choose the Right Claims Data Vendor for Your Organization. 

An experienced partner works through these questions with your team. Since 2010, HDI has integrated data from 627 healthcare data vendors and more than 2,600 vendor layouts across its client base. Through the HDI Data Management Model, HDI acquires each source, loads it into the HDI Core Data Model for transformation, and applies more than 300 business rules and validations that link reversals and corrected claims to the original claim. HDI then delivers the data in the format each downstream target requires for a single source of truth across systems. HDI also monitors each source to help catch layout changes early, before they reach downstream reports.

Healthcare data standards give health systems and payers a shared starting point. Reliable claims data depends on knowing how each source departs from them and managing those differences with each delivery, so your team can spend its time on analytics instead of translation.

How does your current approach to payer data hold up? The five questions above are a starting point. HDI’s External Data Management Assessment helps you score how well building internally, using an analytics platform, or working with a specialist partner fits your organization, and gives recommendations based on your answers.