Script News Download

Avatar
Ulises Cune
  • updated

Hello Community,

I have created this Python script that downloads the news from the site www.forexfactory.com and saves it in the directory (C:\Users\your_name\AppData\Roaming\MetaQuotes\Terminal\Common\Files\News\FF\*. csv) that uses CP.

It does not insert the values of "revised" and "revised_time.", the rest of the values are the same as Andrey has in Google Drive.

You can read more about the News Filter

The news are downloaded from:

Image 3258

You need to download Python 3.x and install the plugins run "pip3 install beautifulsoup4" and all others that you do not have installed

Image 3260


CommunityPowerEA_News_Download.py

Replies 50
Avatar
1
th0mas51

Please find attached a slightly modified version of the script.

I have fixed the problem I talked in my previous post, and I added couple of improvements that I needed on my side at the same time, in case other people find that useful.

I added couple of parameters to allow a more flexible usage of the script:
1) --force: allow to re-download the last 2 month of data
2) --utc: allow to download a new set of data using UTC timezone, the files will be suffixed with _UTC in their name so they don't impact the default behavior of the script, which uses NewYork timezone by default
3) --next_week: allow to download next week's data. Useful for people that trade pairs that can have news on Monday morning/night and want to stop their positions couple of hours before (i.e. on Friday)

You can see the changelog in the header of the script

Feel free to ignore my changes if you don't like them :)


HistoryLoad_v2_2024-01-06.py

Avatar
0
th0mas51
I'm working on some other improvements, namely generating files that are ready for Optimization Backtest in the MQL Cloud or using remote computers.

The idea is that the EA would send those CSV files to the remote Testers agent so the News filter can work in Optimized Backtest.

Out of curiosity, do we really need the values for each news ?

I doubt anyone is using the values, as they are never known in advance, and the values does not necessarily predict in which direction the price is going to move.

I think we could save space and simplify the CSV files by not bothering storing the values.
Avatar
0
th0mas51

After some verification, it look like the time syntax for "10th-15th" does not mean between 10:00 and 15:00 but means between the 10th day of the month and the 15th.

If we look at the notes on FF, this is what we can see:


Image 5609


I guess we have two choices we should probably tag this news as an "All day event", unless FF is able to update the date/time when we get closer to the date.

Avatar
0
Ulises Cune

Are you talking about future news, do you have future candles to backtest?

Avatar
1
th0mas51

Sorry, I realize now that what I said was confusing.

I know that CP does not use this script for Live trading, but I'm basically trying to have a single script that is able to cover the CSV download for both Backtest (in CP and in my own EA) and Live trading (in my own EA).

Since this script is working nicely to download historical News data for CP, and since I'm trying to have a single News downloader script to maintain, I added this "next week" function to the script to be able to cover some of my needs related to Live trading data in my EA.

Basically most of the EAs are not able to know next week's news, which makes overnight trading tricky on Friday, because the EAs doesn't know what news will be happening on Monday morning, and it will only discover that on the first tick of the day, on Monday, right after it has downloaded the most recent News data of the new week.

This limitation is mostly because FF is only offering the current week's news by default (on their ICS, CSV, JSON and XML endpoints), and it would be pretty complex to fetch next weeks news from MetaTrader by parsing the whole HTML code.

So in order to bypass this "next week" data missing limitation for my own EA, I'm delegating the News downloading outside of the EA, to this python script running periodically in the Windows Task Scheduler, and then my EA only has to read the CSV file locally, and always has access to next weeks' News data.

I might be wrong, but I'm assuming that CP is impacted by this same limitation of not being able to know next week's News data on Friday, and has to wait Monday morning to get the news.

Anyway, this tiny feature is mostly for people like me that want to have a single source of News data as CSV, and use it both for CP backtesting and for their other EAs.

Avatar
0
Ulises Cune

Tell me what the candlestick patterns of the future will be so that you can use future news, and we will all be millionaires.

Avatar
0
th0mas51

Haha, I wish it was as easy as looking at next week's calendar and be millionaire :)

As I said, if you don't think that the extra features I added to the script are useful, feel free to not include them in the official version.

I can even delete the file I have attached in my previous post if you think those extra features are not useful, I won't be mad :)

If we delete my version of the script, at the very least we can copy my fix for the bug with the "10th-15th" time syntax.

Avatar
0
Ulises Cune

I have seen the changes, the improvements are appreciated, it is now available to everyone, thank you very much.

Avatar
0
Jan Corsair

Great! Old news...

What about future news?


;-)

Avatar
0
th0mas51
Quote from Jan Corsair

Great! Old news...

What about future news?


;-)

I'm not exactly sure what you mean by that.

I believe that most people don't use the numbers (previous, forecast, actual), as the only thing that really matter is the calendar schedule (date, time, type of event and impact)

In that sense, the latest version of the script is already able to export next week's calendar schedule, which should be enough.