"The index file is older than the data file" when opening 1000 Genomes VCF

[PlatonB]

elementary OS 5.1
pysam 0.15.3

1. Download any 1000 Genomes VCF:
   ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz

2. Download corresponding index:
   ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz.tbi

3. Try to create a VariantFile object:
   from pysam import VariantFile
variant_file_obj = VariantFile('/path_to/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz')

[W::hts_idx_load2] The index file is older than the data file: /path_to/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz.tbi
[W::hts_idx_load2] The index file is older than the data file: /path_to/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz.tbi

There was no error in the previous version of pysam.

[PlatonB]

Pysam 0.15.4: bug is actual.

[jmarshall]

That warning message is from htslib, and has been in pysam's bundled htslib since 2014. It has not changed. Presumably when you saw no message from previous versions of pysam, your previous copies of these files had different timestamps.

What are the timestamps on the /path_to/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz* files on your machine? I expect the warning is accurate. You should touch or regenerate the .tbi file to resolve it.

The timestamps for these files on ftp.1000genomes.ebi.ac.uk make it appear that the .tbi index may be out of date:

$ curl -I ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz
Last-Modified: Fri, 05 May 2017 14:26:27 GMT
Content-Length: 1061765413
Accept-ranges: bytes
$ curl -I ftp://ftp.1000genomes.ebi.ac.uk/vol1/ftp/release/20130502/supporting/GRCh38_positions/ALL.chr6_GRCh38.genotypes.20170504.vcf.gz.tbi
Last-Modified: Fri, 05 May 2017 13:50:07 GMT
Content-Length: 161964
Accept-ranges: bytes

so if your downloaded files reflect these timestamps (e.g., you used curl -R) then pysam will correctly produce this warning.

[PlatonB]

Thanks for the explanation.