Data model

IDC relies on the DICOM data model for organizing images and image-derived data. At the same time, IDC includes certain attributes and data types that are outside of the DICOM data model. The Entity-Relationship (E-R) diagram and examples below summarize a simplified view of the IDC data model (you will find the explanation of how to interpret the notation used in this E-R diagram in this page from Mermaid documentation).

IDC content is organized in Collections: groups of DICOM files that were collected through certain research activity. We sometimes refer to these as Original Collections to distinguish them from Analysis Results collections described below.

Collections are organized into Programs, which group related collections, or those collections that were contributed under the same funding initiative or a consortium. Example: TCGA program contains TCGA-GBM, TCGA-BRCA and other collections. You will see Collections nested under Programs in the upper left section of the IDC Portal. You will also see the list of collections that meet the filter criteria in the top table on the right-hand side of the portal interface.

Individual DICOM files included in the collection contain attributes that organize content according to the DICOM data model.

Each collection will contain data for one or more cases, or patients. Data for the individual patient is organized in DICOM studies, which group images and derived objects collected in the context of a single imaging exam or encounter. Studies are composed of DICOM series, which in turn consist of DICOM instances. Each DICOM instance corresponds to a single file on disk. As an example, in radiology imaging, individual instances will most often correspond to image slices in multi-slice acquisitions (although a single instance of an enhanced multi-frame object, or of a Segmentation, can hold an entire volume), and in digital pathology you will see a separate file/instance for each resolution layer of the image pyramid, plus any label and overview images. Instance is not a level you browse in the IDC Portal - you will encounter individual instances once you download data to your computer.

PatientID is unique within a collection, but is not guaranteed to be unique across IDC. If you join or group data across collections, always use PatientID together with collection_id.

Analysis results

The Analysis Results collection is a very important concept in IDC, and the peer of the Original Collection introduced above. An analysis result - we use the shorter form from here on - is the DICOM encoded result of some analysis performed on data from one or more original collections. Such analysis results are often contributed by investigators unrelated to those that submitted the analyzed images, and may span images across multiple collections.

An analysis result is a collection in its own right. It has its own identifier, title, DOI, license, description and citation, recorded in analysis_results_index just as original collections are recorded in collections_index. That grouping is what establishes the provenance of the derived content and gives credit to the people who produced it - they are cited for their contribution, and their terms of reuse travel with it.

IDC's metadata treats the two as peers explicitly. A collection is composed of one or more contributing sources, listed in the sources field of collections_index. Each entry carries its own source_doi, license, citation and source_title, and is tagged with a source_type of either original_data or analysis_result. For an analysis result, the source_id in that list is its analysis_result_id.

What is different is where the content sits. An analysis result brings no images, patients or studies of its own; it enriches data that is already in IDC. So each derived series belongs to two collections at the same time, and IDC records both:

• collection_id - the original collection that supplied the analyzed images. A derived series carries this exactly like the images it describes, which is what places it in the right patient and study.

• analysis_result_id - the analysis result that contributed it. Set for derived series only; NULL for originally submitted images.

The two are orthogonal grouping axes, not a hierarchy: an analysis result is not nested under one original collection - tcga_sbu_til_maps spans 23 of them - which is why the IDC Portal offers analysis results as a search scope of their own, alongside programs and collections. Filter on whichever axis you actually mean.

A collection_id filter returns both axes at once. Derived series carry the collection_id of the images they analyze, so filtering on a collection returns the analysis results contributed to it alongside its original images - and there can be far more of those than you expect. collection_id = 'nlst' matches 590,572 series, but only 204,346 of them are NLST's own images; the other 386,226 were contributed by analysis results. 55 of the 176 collections contain derived series.

• Original images only: add analysis_result_id IS NULL. That is SQL, so it applies wherever you query index with SQL - idc-index, POST /v3/sql, the MCP run_sql tool, or BigQuery. The column is genuinely NULL for original data, not an empty string, so = '' matches nothing. There is no cohort-filter equivalent: analysis_result_id is a filterable term, but its only values are the 24 analysis result ids, so no term means "none".

• One analysis result: filter on its analysis_result_id - which does work on every surface - rather than on the collections it spans. tcga_sbu_til_maps contributed 21,030 series across 23 collections; selecting those 23 collections instead returns 66,878 series, most of them original imaging you did not ask for.

Two properties of derived content follow from all this:

• Analysis results do not introduce new patients, and almost always attach to a study that already exists, alongside the images they describe - which is why you see them overlaid when you open the study in the viewer. As of v24, every patient with derived series also has original imaging, and only 260 of the ~92,000 studies containing derived series consist of derived series alone.

• Provenance and licensing follow the contributing source, not the collection: series within a single study can carry different source_DOI and license_short_name values, which is why the diagram above places both attributes on SERIES. Derived content is not the only reason - a collection can have several original sources too, as NLST does with its CT images (CC BY 4.0) and its DICOM-converted slide microscopy (CC BY 3.0). As of v24, 37 of the 176 collections have more than one original source. License information is available programmatically at series granularity - see Licensing and attribution.

The model on a concrete example

The specific counts and version numbers below are as of IDC data release v24, and will change as new data is added.

Radiology: PROSTATEx

Consider patient ProstateX-0217 from the PROSTATEx collection (open in the IDC viewer):

PROGRAM  community
└── COLLECTION  prostatex  (PROSTATEx)
    └── CASE  ProstateX-0217
        └── STUDY  1.3.6.1.4.1.14519.5.2.1.7310.5101.239746591836843122771107560214
            │
            ├── 43 original MR series    DOI 10.7937/k9tcia.2017.murs5cl   CC BY 3.0   since v2
            │   ├── t2_tse_tra                     23 INSTANCEs (one file per slice)
            │   ├── t2_tse_sag                     19 INSTANCEs
            │   ├── diffusie-3Scan-4bval_fs        60 INSTANCEs
            │   ├── 35 × "tfl_3d dynamisch fast"   12 INSTANCEs each (DCE time points)
            │   └── …
            │
            └── 4 series contributed by ANALYSIS_RESULTs, added to this same study:
                ├── SEG  prostatex_seg_zones     10.7937/tcia.nbb4-4655       CC BY 3.0   since v2
                ├── SEG  prostatex_seg_hires     10.7937/tcia.2019.deg7zg1u   CC BY 3.0   since v2
                ├── SEG  bamf_aimi_annotations   10.5281/zenodo.8345959       CC BY 4.0   since v19
                └── SR   prostatex_targets       10.5281/zenodo.15643312      CC BY 4.0   since v23

CLINICAL_DATA for ProstateX-0217, across 4 of the 6 clinical tables for this collection:
    prostatex_images  8 rows │ prostatex_findings  1 │ prostatex_ktrans  1 │ bamf…qa_results  1

What this single patient illustrates:

• A derived series belongs to two collections at once. All four carry collection_id = 'prostatex', just like the MR images, and the analysis_result_id of the analysis result that contributed them - which is where their DOI and license come from.

• They attach to the existing study. No new patient, no new study - the SEG and SR objects land in the same StudyInstanceUID as the images they describe.

• One study can carry several licenses and DOIs. Here, three DOIs under CC BY 3.0 and two under CC BY 4.0. Licensing and provenance attach at the series level.

• A study accretes content over releases. The images arrived in IDC v2, the BAMF segmentation in v19, the lesion annotations in v23.

• SeriesDescription is not an identifier. 35 series here share the description tfl_3d dynamisch fast; only SeriesInstanceUID distinguishes them.

• A case maps to many clinical records, spread across several tables and joined on dicom_patient_id - and one of those tables was contributed by an analysis result rather than by the original submitters.

Digital pathology: TCGA-LUAD

The same model applies to slide microscopy, where the INSTANCE level looks quite different. Patient TCGA-80-5608 from the TCGA-LUAD collection (open in the SLIM viewer):

PROGRAM  tcga
└── COLLECTION  tcga_luad  (TCGA-LUAD)
    └── CASE  TCGA-80-5608
        └── STUDY  2.25.67565461533469433863078107064394326180
            │
            ├── SM series "FFPE HE TP DX1"   DOI 10.5281/zenodo.12689915   CC BY 3.0   since v8
            │   └── 4 INSTANCEs, one per pyramid level:
            │       21987 × 17849  (0.50 µm/px, base layer, 128 MB)
            │        5496 ×  4462  (2.00 µm/px)
            │        2748 ×  2231  (4.00 µm/px)
            │         946 ×   768  (11.6 µm/px, thumbnail)
            │
            ├── ANALYSIS_RESULT  pan_cancer_nuclei_seg_dicom   CC BY 4.0
            │   ├── ANN series  1 INSTANCE   (nuclei as bulk annotations)   since v19
            │   └── SEG series  4 INSTANCEs  (nuclei as segmentations)      since v20
            │
            └── ANALYSIS_RESULT  tcga_sbu_til_maps             CC BY 4.0   since v23
                ├── SEG  "Stony Brook CNN-generated TIL Map"
                ├── SEG  "Stony Brook Inception-V4 Binary TIL Map"
                └── SEG  "Stony Brook Inception-V4 Fractional TIL Map"

Two additional points this example makes:

• In pathology, one instance is one pyramid layer, not one slice. The whole slide image above is a single series of 4 files, ranging from a 128 MB base layer at 0.5 µm/px down to a thumbnail. (A small number of series in IDC split a single layer across several instances.)

• A single analysis result can span many collections. tcga_sbu_til_maps covers 23 TCGA collections and pan_cancer_nuclei_seg_dicom covers 14 - this is the many-to-many relationship between ANALYSIS_RESULT and COLLECTION in the diagram above. A single analysis result can also contribute more than one type of object, as pan_cancer_nuclei_seg_dicom does with ANN and SEG series.

Where each entity lives in the metadata

The table below maps each entity to the identifier you would use in idc-index. The same DICOM identifiers are used in the BigQuery dicom_all table, which additionally exposes every other DICOM attribute - see Files and metadata.

Entity	Identifier	Where to find it
PROGRAM	program_id	collections_index
COLLECTION	collection_id	index, collections_index
CASE	PatientID	index
STUDY	StudyInstanceUID	index
SERIES	SeriesInstanceUID	index
INSTANCE	SOPInstanceUID	BigQuery dicom_all; sm_instance_index for slide microscopy
ANALYSIS_RESULT	analysis_result_id	index, analysis_results_index
CLINICAL_DATA	dicom_patient_id	per-collection clinical tables, see clinical.md

Note that the identifiers above are the ones you search with. The files themselves are named using IDC-assigned UUIDs (crdc_series_uuid and crdc_instance_uuid) so that IDC can support versioning - see UUIDs and GUIDs.