Downloads
In addition to the HTTP Archive source code being open source,
all of the data gathered is available as downloadable MySQL dumps.
These dumps are also available in Google BigQuery thanks to Ilya Grigorik .
For more information see Ilya's blog post HTTP Archive + BigQuery = Web Performance Answers
and video .
And checkout all the custom queries shared on discuss.httparchive.org .
MySQL Dumps
The results from each crawl are saved as MySQL dump files in both MySQL format and CSV format.
Dumps are made for both the desktop and mobile crawls.
The dump files do not contain the commands to create the MySQL database and tables.
To restore a dump:
Import the schema to create the tables.
cat httparchive_schema.sql | mysql -u MYSQL_USERNAME -p -h MYSQL_HOSTNAME MYSQL_DB
Import the desired dump file. Use this command for MySQL-formatted dumps:
gunzip -c MYSQLDUMP_FILE.gz | mysql -u MYSQL_USERNAME -p -h MYSQL_HOSTNAME MYSQL_DB
For CSV-formatted dumps, gunzip the file and rename it to either pages.csv
or requests.csv
, then run this command:
mysqlimport --local --fields-optionally-enclosed-by="\"" --fields-terminated-by=, --user=MYSQL_USERNAME -p MYSQL_DB MYSQLDUMP_FILE.csv
If you want to run a private instance of the source code, you need to also import the stats and crawls dumps.
Files
These files define the schema and the meta-level tables:
schema - the schema for the tables referenced in the data dumps
stats - the aggregated stats for all crawls
urls - the URLs used in the most recent crawl
crawls - meta-information about all of the crawls
There's a download file for each type of crawl:
Nov 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
May 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Apr 1 2019: Chrome Android: pages (mysql , CSV ), requests (mysql , CSV )
Mar 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Feb 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jan 1 2019: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
May 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
May 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Apr 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Apr 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Mar 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Mar 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Feb 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Feb 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jan 15 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jan 1 2018: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 16 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
May 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
May 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Apr 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Apr 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Mar 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Mar 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Feb 15 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Feb 1 2017: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jan 15 2017: Chrome Android: pages (mysql , CSV ), requests (mysql , CSV )
Jan 1 2017: Chrome Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Dec 2 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Nov 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Oct 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Sep 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Aug 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jul 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV )
Jun 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jun 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
May 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
May 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Apr 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Apr 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Mar 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Mar 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Feb 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Feb 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jan 15 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jan 1 2016: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Dec 15 2015: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Dec 1 2015: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Nov 15 2015: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Nov 1 2015: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Oct 15 2015: Chrome: pages (mysql , CSV ), requests (mysql , CSV ) Android: pages (mysql , CSV ), requests (mysql , CSV ) IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Oct 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Sep 15 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Sep 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Aug 15 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Aug 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jul 15 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jul 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jun 15 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
Jun 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
May 15 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
May 1 2015: IE: pages (mysql , CSV ), requests (mysql , CSV ) iPhone: pages (mysql , CSV ), requests (mysql , CSV )
sponsors :
Google ,
Mozilla ,
New Relic ,
O’Reilly Media ,
Etsy ,
dynaTrace ,
Instart Logic ,
Catchpoint Systems ,
Fastly ,
SOASTA mPulse ,
Hosting Facts ,
TheBestVPN ,
and
Akamai
.
powered by WebPagetest