Dark Theme
Light Theme
Trading Articles
Article Categories
Article Tools
Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to
register in order to view the content of the threads and start contributing to our community.
It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Yahoo Finance - historical daily data retrieved programmatically
Updated April 16, 2014
trending_up
11,549 views
thumb_up
5 thanks given
group
2 followers
forum
5 posts
attach_file
0 attachments
Yahoo Finance - historical daily data retrieved programmatically
April 16th, 2014, 10:29 AM
near Paris, France
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769
Hi,
In this thread, I will propose several ways to retrieve historical data from Yahoo Finance :
- Web Query (from Java code),
- YQL (from Java code),
- Quantmod (with R).
If you have other or better ideas, do not hesitate to contribute.
EDIT: Yahoo Finance EOD data is available through Ninja Trader : refer to @ratfink 's message below.
Nicolas
Can you help answer these questions from other members on NexusFi?
Best Threads (Most Thanked) in the last 7 days on NexusFi
April 16th, 2014, 10:38 AM
near Paris, France
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769
1) Web query and related Java code
Let's consider the following URL :
https://ichart.yahoo.com/table.csv?s=^GSPC&a=3&b=1&c=2014&d=3&e=15&f=2014&g=d
It generates a csv file for ^GSPC, that is to say S&P 500, with daily data (g=d) from April 1st, 2014 (a=3&b=1&c=2014) to April 15th, 2014 (d=3&e=15&f=2014).
More information on the syntax here : https://code.google.com/archive/p/yahoo-finance-managed/wikis/csvHistQuotesDownload.wiki
Other examples of symbols:
MSFT for Microsoft
EN.PA for French 'Bouygues' stock
However, it seems that no historical data is available for future. Refer to python - Download future price series from Yahoo! with Pandas - Stack Overflow
In the CSV file, last data is Adjusted Close , which takes into account split and dividend .
References: https://uk.help.yahoo.com/kb/finance/historical-prices-adjusted-close-sln2311.html?impressions=true and https://help.yahoo.com/kb/finance/historical-prices-sln2311.html?impressions=true
Example of Java code:
Code
import java.io.BufferedReader;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
// inspired by http://stackoverflow.com/questions/9093000/read-csv-file-from-internet
public class Yahoo_ReadCSVFromWebQuery {
public static void main(String[] args) {
try {
URL url = new URL("http://ichart.yahoo.com/table.csv?s=^GSPC&a=3&b=1&c=2014&d=3&e=15&f=2014&g=d");
URLConnection urlConn = url.openConnection();
InputStreamReader inputStreamReader = new InputStreamReader(urlConn.getInputStream());
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
String line;
while ((line = bufferedReader.readLine()) != null) {
System.out.println(line);
}
bufferedReader.close();
inputStreamReader.close();
} catch (Exception e) {
e.printStackTrace();
}
}
}
It produces the following output:
Quoting
Date,Open,High,Low,Close,Volume,Adj Close
2014-04-15,1831.45,1844.02,1816.29,1842.98,3736440000,1842.98
2014-04-14,1818.18,1834.19,1815.80,1830.61,3111540000,1830.61
2014-04-11,1830.65,1835.07,1814.36,1815.69,3743460000,1815.69
2014-04-10,1872.28,1872.53,1830.87,1833.08,3758780000,1833.08
2014-04-09,1852.64,1872.43,1852.38,1872.18,3308650000,1872.18
2014-04-08,1845.48,1854.95,1837.49,1851.96,3721450000,1851.96
2014-04-07,1863.92,1864.04,1841.48,1845.04,3801540000,1845.04
2014-04-04,1890.25,1897.28,1863.26,1865.09,3583750000,1865.09
2014-04-03,1891.43,1893.80,1882.65,1888.77,3055600000,1888.77
2014-04-02,1886.61,1893.17,1883.79,1890.90,3131660000,1890.90
2014-04-01,1873.96,1885.84,1873.96,1885.52,3336190000,1885.52
April 16th, 2014, 12:53 PM
Birmingham UK
Market Wizard
Experience: Intermediate
Platform: NinjaTrader
Broker: TST/Rithmic
Trading: YM/Gold
Posts: 3,633 since Dec 2012
Thanks Given: 17,423
Thanks Received: 8,425
Don't forget you do get free access to the Yahoo and Kinetic EOD data streams in the free version of NinjaTrader , you could always just use that as your download route if you don't want to run with it.
April 16th, 2014, 01:51 PM
near Paris, France
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769
2) YQL and related Java code
According to Yahoo, "The Yahoo Query Language is an expressive SQL-like language that lets you query, filter, and join data across Web services. With YQL, apps run faster with fewer lines of code and a smaller network footprint."
Documentation: https://developer.yahoo.com/yql/
Let's go to YQL Console: https://developer.yahoo.com/yql/console/
On the left, tick "Show Community Tables".
Then, the following query:
Quoting
select * from yahoo.finance.historicaldata where symbol = "^GSPC" and startDate = "2014-04-01" and endDate = "2014-04-15"
...returns XML code with S&P 500 daily quotes for the 15 first days of April:
Quoting
<?xml version="1.0" encoding="UTF-8"?>
<query xmlns:yahoo="http://www.yahooapis.com/v1/base.rng"
yahoo:count="11" yahoo:created="2014-04-16T17:48:03Z" yahoo:lang="en-US">
<diagnostics>
<url execution-start-time="0" execution-stop-time="33" execution-time="33"><![CDATA[http://www.datatables.org/yahoo/finance/yahoo.finance.historicaldata.xml]]></url>
<publiclyCallable>true</publiclyCallable>
<cache execution-start-time="36" execution-stop-time="36"
execution-time="0" method="GET" type="MEMCACHED"><![CDATA[1dfb58ee222318d70f556698d247d216]]></cache>
<url execution-start-time="36" execution-stop-time="139" execution-time="103"><![CDATA[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]]></url>
<query execution-start-time="36" execution-stop-time="140"
execution-time="104" params="{url=[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]}"><![CDATA[select * from csv(0,1) where url=@url]]></query>
<cache execution-start-time="141" execution-stop-time="141"
execution-time="0" method="GET" type="MEMCACHED"><![CDATA[015e0267fe0040cfc71fc09364faf796]]></cache>
<url execution-start-time="141" execution-stop-time="203" execution-time="62"><![CDATA[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]]></url>
<query execution-start-time="141" execution-stop-time="204"
execution-time="63" params="{columnsNames=[Date,Open,High,Low,Close,Volume,Adj_Close], url=[http://ichart.finance.yahoo.com/table.csv?g=d&f=2014&e=15&c=2014&b=1&a=3&d=3&s=%5eGSPC]}"><![CDATA[select * from csv(2,0) where url=@url and columns=@columnsNames]]></query>
<javascript execution-start-time="34" execution-stop-time="209"
execution-time="174" instructions-used="90942" table-name="yahoo.finance.historicaldata"/>
<user-time>210</user-time>
<service-time>198</service-time>
<build-version>0.2.2430</build-version>
</diagnostics>
<results>
<quote Symbol="%5eGSPC">
<Date>2014-04-15</Date>
<Open>1831.45</Open>
<High>1844.02</High>
<Low>1816.29</Low>
<Close>1842.98</Close>
<Volume>3736440000</Volume>
<Adj_Close>1842.98</Adj_Close>
</quote>
<quote Symbol="%5eGSPC">
<Date>2014-04-14</Date>
<Open>1818.18</Open>
<High>1834.19</High>
<Low>1815.80</Low>
<Close>1830.61</Close>
<Volume>3111540000</Volume>
<Adj_Close>1830.61</Adj_Close>
</quote>
[...]
</results>
</query>
The following Java code sends the same query:
Code
// requires JDOM API: http://www.jdom.org/
import java.io.BufferedReader;
import java.io.ByteArrayInputStream;
import java.io.InputStreamReader;
import java.net.URL;
import java.net.URLConnection;
import java.util.Iterator;
import java.util.List;
import org.jdom2.Document;
import org.jdom2.Element;
import org.jdom2.input.SAXBuilder;
// inspired by http://cynober.developpez.com/tutoriel/java/xml/jdom/#LIII-A
public class Yahoo_ReadFromYQL {
public static void main(String[] args) {
try {
//
// 1. Retrieve the XML file
//
String prefix = "http://query.yahooapis.com/v1/public/yql?q=";
String query = "select * from yahoo.finance.historicaldata where symbol = \"^GSPC\" and startDate = \"2014-04-01\" and endDate = \"2014-04-15\"";
String suffix = "&env=store%3A%2F%2Fdatatables.org%2Falltableswithkeys";
String s = prefix + query.replaceAll(" ", "%20") + suffix;
URL url = new URL(s);
URLConnection urlConn = url.openConnection();
InputStreamReader inputStreamReader = new InputStreamReader(urlConn.getInputStream());
BufferedReader bufferedReader = new BufferedReader(inputStreamReader);
final StringBuffer buffer = new StringBuffer();
String line;
while ((line = bufferedReader.readLine()) != null) {
buffer.append(line);
}
bufferedReader.close();
inputStreamReader.close();
//
// 2. Parse XML file
//
SAXBuilder sxb = new SAXBuilder();
Document document = sxb.build(new ByteArrayInputStream(buffer.toString().getBytes()));
Element query0 = document.getRootElement();
Element results = query0.getChild("results");
List quotes = results.getChildren();
Iterator i = quotes.iterator();
while (i.hasNext()) {
Element quote = (Element) i.next();
System.out.print(" "+quote.getChild("Date").getText());
System.out.print(" "+quote.getChild("Open").getText());
System.out.print(" "+quote.getChild("High").getText());
System.out.print(" "+quote.getChild("Low").getText());
System.out.print(" "+quote.getChild("Close").getText());
System.out.print(" "+quote.getChild("Volume").getText());
System.out.println(" "+quote.getChild("Adj_Close").getText());
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
It returns:
Quoting
2014-04-15 1831.45 1844.02 1816.29 1842.98 3736440000 1842.98
2014-04-14 1818.18 1834.19 1815.80 1830.61 3111540000 1830.61
2014-04-11 1830.65 1835.07 1814.36 1815.69 3743460000 1815.69
2014-04-10 1872.28 1872.53 1830.87 1833.08 3758780000 1833.08
2014-04-09 1852.64 1872.43 1852.38 1872.18 3308650000 1872.18
2014-04-08 1845.48 1854.95 1837.49 1851.96 3721450000 1851.96
2014-04-07 1863.92 1864.04 1841.48 1845.04 3801540000 1845.04
2014-04-04 1890.25 1897.28 1863.26 1865.09 3583750000 1865.09
2014-04-03 1891.43 1893.80 1882.65 1888.77 3055600000 1888.77
2014-04-02 1886.61 1893.17 1883.79 1890.90 3131660000 1890.90
2014-04-01 1873.96 1885.84 1873.96 1885.52 3336190000 1885.52
April 16th, 2014, 02:10 PM
near Paris, France
Experience: Beginner
Platform: -
Trading: -
Posts: 1,071 since Aug 2011
Thanks Given: 2,232
Thanks Received: 1,769
3) R's quantmod
The following R code
Code
library(quantmod)
SnP500 <- getSymbols("^GSPC", src="yahoo", from=as.Date("2014-4-1") , to=as.Date("2014-4-15"), auto.assign=FALSE)
print(SnP500)
...returns:
Quoting
GSPC.Open GSPC.High GSPC.Low GSPC.Close GSPC.Volume GSPC.Adjusted
2014-04-01 1873.96 1885.84 1873.96 1885.52 3336190000 1885.52
2014-04-02 1886.61 1893.17 1883.79 1890.90 3131660000 1890.90
2014-04-03 1891.43 1893.80 1882.65 1888.77 3055600000 1888.77
2014-04-04 1890.25 1897.28 1863.26 1865.09 3583750000 1865.09
2014-04-07 1863.92 1864.04 1841.48 1845.04 3801540000 1845.04
2014-04-08 1845.48 1854.95 1837.49 1851.96 3721450000 1851.96
2014-04-09 1852.64 1872.43 1852.38 1872.18 3308650000 1872.18
2014-04-10 1872.28 1872.53 1830.87 1833.08 3758780000 1833.08
2014-04-11 1830.65 1835.07 1814.36 1815.69 3743460000 1815.69
2014-04-14 1818.18 1834.19 1815.80 1830.61 3111540000 1830.61
2014-04-15 1831.45 1844.02 1816.29 1842.98 3736440000 1842.98
quantmod::adjustOHLC ( adjustOHLC {quantmod} | inside-R | A Community Site for R ) could be considered to take into account splits and dividends.
This is the end of this series.
Nicolas
Last Updated on April 16, 2014