Dear Francis
Your new siteini file for Starhub Singapore has a problem that several channels will be off by 24 hours, eg I have to look at tomorrows schedule to find todays schedule. Not at all practical when you do recordings based on the program schedule, and actually worse than having no schedule at all :)
I did some research and see that you were using "site {firstshow=1}" in the siteini file. I looked at the data returned by the URL you are grabbing and saw that indeed it returns *one* show from the day before (before midnight), unless the show begins exactly at midnight. So firstshow=1 should take care of that, but it does not. I've played around with different values (0,1,2,now) and it seems to make no difference whatsoever, a lot of channels are off by 24 hours. And I find the first show from todays date in my resulting xml file no matter what value firstshow is set to, seems to me like it has no effect at all, or at least my perception is that it makes no difference.
I'm using beta release 1.1.1/53.08.
I tried the newest beta on your site as well, 1.1.1/53.15, but that one chokes with the following error message:
File ........\guide.xml not found, creating a new one
Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array at WebGrab.Program.Mail(String[] args)
This was without an guide.xml, deleted it between each testrun to be sure I started over with no guide data. Did not try this latest beta build with an excisting guide.xml file. But there might be a problem, probably unrelated to my problem, with the latest beta build as well.
Please, could you have a go at the offset problem with the latest Starhub siteini files? A good candidate for testing would be channel 601 HBO, it always gives me the 24 hours offset problem.
To me it looks like there is a problem/bug with the "firstshow=" parameter as it makes no difference at all what I set it to.
You are here
Some Starhub Singapore channels off by 24 hours
Fri, 2014-02-21 19:54
#1
Some Starhub Singapore channels off by 24 hours
Brought to you by Jan van Straaten
Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Supported by: servercare.nl
Correct.
Tested it here and also not working.
The firstshow=1 is correct, so no need to change.
It has something to do with the timezone. When I change it to +00:00, it works. So probably it has something to do with that.
But as you can seen at the bottom of this page:
Program Development - Jan van Straaten ------- Web design - Francis De Paemeleere
Jan? (Will take some time, because Jan is currently bussy with other stuff, so be patient)
Hi,
as Francis already indicated, I have to finish some updates of the program first but will look into it as soon as possible after that.
Jan
Thanks for the very fast response, hope there will be a reasonable quick solution to this.
And where is that donate button I keep nagging about!!! :=)
Come to think of it. Francis you mentioned that if you set the timezone (I assume in the siteini file) to +00 it works. Maybe I could do exactly that and then use the tool on your webpage (xmltv_time_correct) to adjust the time for those channels that would be offset wrongly?
Do you think that would work that work as a workaround until the bug is fixed?
(My wife is demanding a solution right now, she's not patient like me..... :)
Come to think of it. Francis you mentioned that if you set the timezone (I assume in the siteini file) to +00 it works. Maybe I could do exactly that and then use the tool on your webpage (xmltv_time_correct) to adjust the time for those channels that would be offset wrongly?
Do you think that would work that work as a workaround until the bug is fixed?
(My wife is demanding a solution right now, she's not patient like me..... :)
Did some more testing:
1) That "xmltv_time_correct" utility did not work so well, it would crash with an error message while changing show dates.
2) The simple solution is often the bes1 - so I downloaded "sed" (stream editor) for DOS and made a simple bat-file that would change the text +0000 back to +0800. This worked, sort of
3) Setting timezone to +0000 does not work correctly either. Now I have a somewhat better guide with most channels correct, but for some channels I now have tomorrows guide instead of yesterdays, not sure which is worse :)
So setting timezone +0000 does not solve the problem with firstshow= either.
Let me know if I can help more with identifying the root cause of this annoying bug.
When do you grab the EPG?
I now grab it and it works fine, even with +08:00. (here it is now 7 AM). Maybe grabbing earlier will "fix" it (But not sure).
And for the shows that show yesterday, 1: what is the value of first show + 2: when does the first show it should grab start.
So if firstshow=1, when does the 2nd show start/stop. Or if firstshow=2, when does the 3rd show start/stop?
3: at what time zone are you in? +08:00?
The only other thing I could think of, is to give your creditcard to your wife, so she can shop, while Jan is looking at this one? (If you did not already "tried" that)
I have a scheduled task that grabs the EGP data at 9:05 am, it used to be set at 7:05, I get the same arror at both times. Asnd I've pretty much grabbed the guide data at all the hours of the day while debugging this, I seem to always have the 24 hour error on one or more channels regardless of when I grab the data.
The channels who ends up showing yesterdays schedule seems to have a show that starts before midnight in the raw data I grab, see this link for an example:
http://schedule.starhubtvanywhere.com/data?c=624&d=20140222
So in this case firstshow=whatever does not make a difference. But it seems to be random if and when this happens, but it always happens with one or more channels. I'm not so sure if settint TZ +0000 made any difference other than the random variations I seem to be getting. So it's nto so easy to answer your questions since the behaviour seems a bot random, setting forstshow to 0, 1, 2 or now does not seem to make any difference other than the same random channels being 24 hours out of sync.
And yes I'm in Singapore at the moment so I'm in the TZ UTC+8.
Appreciate all the help I'm getting with this, please let me know if there is something else I can do.
Right now I'm grabbing with TZ set to +0000 and firsthow=1, this seems to give the best result ("seems" is keyword, it's still a bit random), then I'm changing +0000 to +0800 automatically using sed before WMC imports the xml data.
Hi,
Francis suggested to hand over your creditcard to your wife. I know a better one, hand it over to me ..
Anyhow, just now, I did a first test run with Fox Family. I changed my computer to the +8:00UTC timezone, so it is still Sunday , 23/03/2014, run done at 23:35 just before day change. The results of the shows I found in split index:
grabbed 23/02/2014 23:35 local Singapore time
First shows of showsplit:
{"a":"Bring It On Again","i":"11:05 PM","j":2100,"z":"19427"}|
{"a":"Look Who's Talking Too","i":"12:35 AM","j":4800,"z":"19428"}|
{"a":"Problem Child","i":"1:55 AM","j":4800,"z":"19429"}
Here the first show, Bring It On Again is the only one of 'today'.
If we here apply firstshow=1 , this first one is skipped and all the shows will move to 'today, giving all shows a -24 hour shift!
So, grabbing at the end of the day will cause the problem because the first show is correctly still of 'today'. I will try again in a couple of hours and come back with more. I guess we are victim of the update schedule of the site.
Jan
Sorry cannot do, but I *will* use that donate button once it becomes available :=)
Not sure if I follow you. I did the same grabbing and the first show "Bring it on again" is actually on the 22nd, so that show starts at 11:05 PM on the 22nd. So when firstshow=1 is applied you should end up with only programs for the 23rd, I don't see why this logically should give a -24 hour shift. You can check the official TV guide at tvguide.starhubtvanywhere.com, except you won't see "yesterdays shows. I'm pretty sure my logic is correct here, please correct me if it's not.
I've spent a lot of time checking what the grabber returns for the Starhub channels and it is like this:
- Unless a show starts *exactly* at midninght then *one* show from the day before is returned
- If a show starts *exactly* at midninght then no shows from the previous day is returned
- Shows for the next day is *never* returned
This is the rule no matter when you grab, the result will be the same if you grab the listing at 00:01 or 23:59 or at any time in between. At least that is what I have observed, and I've done a lot of testing on this.
I'm not sure if WebGrabPlus consider midnight "today" or "yesterday", but in any case applying "firstshow=1" *should* fix this so I always end up with a schedule for *today only*, no matter when in the day I do the grabbing, or at least that's my understanding.
If my logic above is correct then there must be an error somewhere because this is not the result I'm seeing.
Thanks again for looking into this! :)
Hi,
it saves me time hunting for an example of a show starting at midnight if you just tell me how the syntax of the time value is :
12:00PM or 00:00AM or whatever?
Thanks Jan
Sure, the syntax seems to be: 12:00 AM
In fact all times are given on the format "HH:MM" then a space and then AM or PM
Example:
{"c":[ {"a":"151" ,"b":"Russia Today" ,"c":"http://img.msv.starhub.com/is.img?format=png&src=/2012/05/18/15895227_151_RussiaToday.png" ,"e":"335" ,"f":"0"}],"p":[ {"a":"News Weekly" ,"i":"12:00 AM" ,"j":1740 ,"z":"41677"}, {"a":"Venture Capital" ,"i":"12:29 AM" ,"j":900 ,"z":"41678"}To further assist with debugging this I did some more testing. I set my siteini file for Starhub back to the latest version on your website, including setting the TZ back to the correct +0800. I then did a new grab which started this morning (24/02) at 0900 UTC+8, that is local Singapore time. All my computers time zones are of course set correctly to TZ UTC+8.
I went through the listings for all the channels and compared with the Starhub official TV Guide, but only for the first 24 hours (today 24/02), and these are the anomalities I discovered:
* Channel 151 RT - shows what appears to be the schedule for yesterday (23/02)
- When I "grab" today (24/2) for that channel (http://schedule.starhubtvanywhere.com/data?c=151&d=20140224) I see that the first show on the 24/2 starts exactly at midningt.
- When I "grab" yesterday (23/2) for that channel (http://schedule.starhubtvanywhere.com/data?c=151&d=20140223) it seems like the listing for that day is incomplete and only covers from 14:00 to 23:29.
* Channel 722 CCTV News - snows what appears to be the schedule for yesterday (23/2)
- When I "grab" today (24/2) for that channel (http://schedule.starhubtvanywhere.com/data?c=722&d=20140224) I see that the first show on the 24/2 starts exactly at midningt.
- When I "grab" yesterday (23/2) for that channel (http://schedule.starhubtvanywhere.com/data?c=722&d=20140223) it seems like the listing for that day is incomplete and only covers from 06:00 to 23:30.
So there seems to be an issue when the first show starts at *exactly* midnight, but it might also be a coinsidence that both the channels that had guide data for the first 24 hours wrong had a show starting exactly at midningh. But I think this is worth looking into.
I then went on to do some random checking on the next 24 hours in the grabbed data from this morning, the guide for tomorrow (25/2). This is what I discovered:
* Channel 605 HBO Hits, all the guide data for the next day (25/2) were wrong, actually the error started with the show that crossed midningh, from that show and onwards todays guide data was repeated, then for the next day it seemed like tomorrows guide data were added. So the scedule for the 24th were reused on the 25th and then on the 26th the guide data for the 25th were used.
* Channel 855 VV Drama, all the guide data for the next day (25/2) were wrong here too, and again the error started with the show crossing midninght, from that show and onwards todays guide data was repeated, then for the next day it seemed like tomorrows guide data were added. So the scedule for the 24th were reused on the 25th and then on the 26th the guide data for the 25th were used. So the same pattern repeated itself for this channel.
* Channel 422 Discovery Channel, all the guide data for the next day (25/2) were wrong here too, but this time ther error did not start with the show crossing midninght, because no show crossed midninght. The error started with the show that started at midnight, from that show and onwards todays guide data was repeated, then for the next day it seemed like tomorrows guide data were added. So the scedule for the 24th were reused on the 25th and then on the 26th the guide data for the 25th were used. So the same pattern repeated itself for this channel.
Take note this last example (422 Discovery Channel) had a show the first day that started exactly at midnight, but it did not cause the first day to be 24 hours off like was the case for Channel 151 RT and Channel 722 CCTV News above, so maybe the show starting exactly at midnight does not play a role here after all. But I still think it's worth looking into, it might cause an error under some special cisrcumstances.
I only checked day 2 and onwards in my schedule for the 2 channels above, but I picked them randomly so I'm pretty sure the pattern will repeat itself for all the channels.
So, it is pretty obvious that my guide data is a mess. Sometimes day 1 (ok, most of the time, but not always) will be ok, but from day 2 onwards it's pretty much a mess.
My gut feeling is that there is a bug somewhere in the software that causes some of this, and I also suspect that the guide data for the first day is somehow grabbed twice.
Hope this is enough information to find and correct those pesky little bugs, please let me know if there is anything else I can do to assist.
Thank you gentlemen! :=)
Hi,
I did some testing with some of the channels. One thing I found is that occasionally there are duplicated shows on the subsequent index pages. I added a few lines to the ini to remove them. I also changed the cultureinifo to en-US to be sure that there is no misinterpretation of the time format used by the site. I checked the result of that with 151 RT (with the first show starting at 12:00AM) and found all in order. No day or time shifts. See the epg picture and the attached xmltv.
I have set my computer to Singapore timezone for the tests.
I also include the changed ini. Please use this to verify my results at your side of the continents.
I will do some more testing with the other channels you provided later today.
Jan
P.S. Regarding the 'donate button' . There is some nice progress there, we have a tryal setup with which only the members of the team can do a donation!! So ,as such a try, I made a donation. I assure you, its a good feeling , but obviously, in my case, it makes little sense to donate to 'myself' and share that with the other members of the team. The problem there ... Francis is simply to busy supporting all of you, too little time left to finish this.
Great news! Francis just mailed me that the donate button is online! (on the home page at the right side, you can't miss it!)
Jan
Thank you, will do some more testing tomorrow and get back to you, it's getting late here now. And it just turned midnight, which means that all my guide data are messed up again.... :)
I can confirm that the "Donate button" is indeed working, I'll be using it once a year or so for as long as I continue to use your software and enjoy your excellent support. Hopefully others too will appreciate free software and realize that "time is not free" so they too will use the button!
Oh and thank you for the screen shot of the XML EPG Viewer, it never crossed my mind to look for one so I've been "reading the XML" and occasionally imported the guide to WMC and used a web browser to look at the guide from there. Now this View makes the process a whole lot easier and less time consuming.
(I should probably use the "donate button" again, because my time is money too and you just saved me some time )
I installed your updated siteini file and all the channels were grabbed again this morning at 0910 Singapore time, I gad deleted the guide.xml to be sure I started from scratch.
I then checked 10 random channels, and they all have the schedule (todays schedule) for today and tomorrw, then the day after tomorrow have tomorrows schedule and so on. So the main problem right now is that the first days schedule is repeated twice and then from then on all challes are 24 hours off because of that.
I see no point in doing any more detailed testing until that is fixed. As soon as that is fixed I'll go detailed through each and every channel and look carefully at all the shows.
Thanks again for looking into this :)
That sound like an older WG++ version (we had that bug some time ago)? What version are you using? Send me your files (log , config)
Jan
As I write in my first post, I'm using beta release 1.1.1/53.08 :)
Attached you'll find my config and latest log file.
PS: I tried the newest beta on your site as well, 1.1.1/53.15, but that one chokes with the following error message:
File ........\guide.xml not found, creating a new one
Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array at WebGrab.Program.Mail(String[] args)
I "upgraded" to a beta release in early january because francis made a change to a ini file that required it.
Hihi,
From "Run patchexe.bat as administrator!.txt"
- bug fixed: index_pages were saved double! Don't use 53.8!!
You should use the latest beta.
We should check what the error is with that one.
duh, bummer! :)
I'll install the latest beta right away. it gave me an error message last time I tried it, but that was with an empty (no existing) guide.xml file. Will try again and report back
Thanks!
OK I installed the latest beta but it just fails with the following error message seconds after starting it:
Unhandled Exception: System.IndexOutOfRangeException: Index was outside the bounds of the array at WebGrab.Program.Mail(String[] args)
My configuration files are those I posted a couple of posts up in this thread. The logfile does not give any further clues as to why the program crash:
I'' try to revert back to the latest official build while you're figuring this out. Please let me know if I can do anything else to assist with this.
Thank you both!
I had a previous beta build 1.1.1/53.07 laying around so I reverted back to that build and started a new grab. Will check the result and report back tomorrow as it's getting late here now. Thanks again!
After going back to version 1.1.1/53.7 I *still* get duplicate schedules for the two first days for all my channels, and the rest of the days is then 24 hours offset. Does the same bug exist in this version as in 1.1.1/53.8?
And version 1.1.1/53.15 is just crashing, see above.
I guess I'll have to wait for a new beta release that works and see if that fix the problem.
Any other suggestions?
Hi again,
I really pity you!! The reason of the runtime error with 53.15 is that it uses a brand new config parser that behaves a little different from the one in 53.07. In your config file the <postprocess> was completely commented -out, and the new parser didn't like that and crashed. Please change that in your config file by adding
<postprocess/>
Of course the next build will run with the disabled postprocess the way you specified it.
Another thing : I don't recomment to use the update mode s (smart). It is rarely used and in fact a left-over from a predecessor of the program only added for compatibilty reasons. The best update mode is i (incremental) of if you really want to start with an empty guide use f (full).
Jan
P.S. Thanks for your generous contribution.
I thought I posted some preliminary results a few hours ago but the post is no longer here, oh well I was tired so maybe I made some mistake.
Anyhow thanks a million, all seems to be well now. I installed beta 1.1.1/53.15 and changed the config.xml as suggested. I also changed the "maybe-not-so-smart" grabbing mode to incrimental as recommended. I did a new grab this morning and I checked all the channels for all the days in the shedule.
No more dupicate days and no problems around midnight, the resulting xmldata is exactly like the official Starhub guide online. perfect, thanks for all the help and guidance!
I'm now heading over to the "Mdb & Rex" section for a few more questions, please bear with me... :)