I'm currently working on RHC rhef.com.au which is proving to be a tricky one. It looks as if I require POST data as the guides load in a unique DIV with javascript links. Attached is what I have so far. If you have any tips for me I'd love to know them!
Attachments:
Hi,
you were very close with the url_index setup. Try this:
site {url=rhef.com.au|timezone=UTC+09:30|maxdays=7|cultureinfo=en-AU|charset=utf-8|titlematchfactor=90}
url_index{url(debug)|http://www.rhef.com.au/wp-admin/admin-ajax.php}
url_index.headers {method=POST|contenttype=application/x-www-form-urlencoded}
url_index.headers {postdata=action=switch_guide_main&date='urldate'}
urldate.format{datenumber|UNIX|0}
With this I got the proper page. I haven't looked at the rest as yet, may be you can try that. If not, I will help.
There is a small problem though : The site 'officially' disallows grabbing! I don't understand why because that normally happens now and then with commercial sites, which is not the case here I presume. If they want to limit access because of server load, that also doesn't make sense because they have online video which in serverload is way more than the few seconds it takes for WG++ to grab the schedule. It must be in their interest to support viewers also with allowing them to download the schedules in the format they can use . Maybe you can contact them and explain. If they are willing to allow WebGrab+Plus we can set the user-agent to WebGrab+Plus (in the config) to identify our grabbing. They can then allow Webgrab by adapting the robots.txt content.
You can also just ignore the warning and continue without their agreement. It is not illegal or so.
Jan
ps I will have a look at the ictv problem , but I am a little busy right now
No problems and thanks again for your help.
I have made some progress with this. I'm able to get some start times and titles, but only from one block/element per page. There also seems to be an issue with the dates.
Regarding the copyright issues,
I'm not too worried. The foundation was supplying me with TV guides personally via email to help with an XMLTV project, but I stopped it a few months ago because it was too hard to manually convert these to XML every week. The current website has only existed for a couple of months and the previous website never had a copyright notice.
I'll follow your suggestion and contact them. Thanks :)
Hi Smacca,
I have completed the rhef.com.au.ini, you can find it @http://www.webgrabplus.com/sites/default/files/download/ini/info/zip/Australia_rhef.com.au.zip
For the description you might want to add or change things,( the site has lots of data that could be considered as description).
You again need to install the beta (51.3) http://www.webgrabplus.com/sites/default/files/patchexe_prebuild.zip, I found another small bug which sets the wrong datenumber with high values of timezone.
Jan
PS Does the site still provide xml (or other format) feeds? It is very easy to use WG++ as xmltv converter.
PS You can temporarely disable the 'disallow' message by removing Disallow: /wp-admin/ from the file www.rhef.com.au.robots in the robots folder. This file is updated once a month so you have to repeat that now and again.
Saving the day once again, Jan. Thank you very much for this.
I'll have a play with the description and see what works. There is however a problem with the dates. It's scrubbing them 24hrs ahead. This channel isn't in +09:30, so I changed the timezone to +10:00. I get the same results on either timezone.
Another minor issue is the gaps between shows. For example, a show finishes at 2:27pm and next show starts at 3:00pm. Most HTPC's would handle this, but Is there a way to fill these gaps or round finish times to nearest 10 mins anyway? For the record, RHC always uses specific program blocks, ie Mon-Fri 13:30-15:30, 19:30-21:30, Sat off-air and Sun 16:30-18:00.
The admins at RHC provided a weekly email with a Word document which I couldn't convert to anything. It was a favour they were doing to be part of a wider TV guide service which I maintain - see http://www.sattvguide.com.au/ and http://www.sattvguide.com.au/xml/ I didn't have the time to manually enter every show. Since then, this new website has started and here we are now. :)
re date error : is your computer set to 'somewhere in Australie' time? What is the timezone setting of you computer? You are using build 51.3 I presume? Can you check the urldate values in the postdata (add debug to url_index , run, look in the logfile for the generated values.)
re stoptimes : I use duration plus start time. With some effort it could be changed to next start time if not between 15:30 and 19:30 (then stoptime=15:30) and after 21:30 (then stoptime=21:30). Something similar for sunday. Also a dummy show have to be added for saterday because the program doesn't like full day gabs. Complicated be doable, I think.
Anything else??
For whom are you running the TV Guide service?
Jan
I'm sorry, I mistook the Saturday problem as a general date problem. With Saturday missing, it puts all the dates out for Sun, Mon, Tue, etc, so it appears my timezone settings and everything else is correct (I am in +08:00). Yes, I have upgraded to latest prebuild.
I'll have a play with modfying the stop times and see how far I get.
Sat TV Guide is for a rural satellite free-to-view TV service called VAST. There are no internet TV guides that are accurate enough for these viewers, so I created Sat TV Guide to fix the problem. I mostly do it for a visual browser guide but I also offer XML files for anyone interested. I started off using siliconhill data, but once I found WebGrab I was very happy. Currently using a mixture of YourTV and OzTiVo data.
Thanks again Jan :)
Hi,
I added dummy shows in the 'empty' afternoon and evening blocks on Saterday and Sunday (or on any othe day with empty blocks). This solves the date problem. If you don't like the title "No showing at this time" you can change that in the ini.
http://www.webgrabplus.com/sites/default/files/download/ini/info/SiteIni.Pack/Australia/rhef.com.au.ini
I haven't done anything with the stoptime issue. Do you really feel that as a problem? (It's not so easy to fix because there are a lot of conditions to take into account)
Glad to be of help ..
Jan
Hi again Jan,
My apologies for not replying sooner. Unfortunately RHEF ceased broadcasting later in 2013, so this ini is no longer relevant.