I want to automate grabbing from tvtv.us
I understand that tvtv.us requires the location to be set by visiting the website manually. This works fine until the cookie it sets expires, I then have to visit the website manually to set the location again.
Is there a way to automate this so I don't have to keep visiting the website manually and resetting the location?
There is a whole thread on tvtv.us about cookies, probably you do not follow the instructions, it needs to be empty and read-only with:
User-agent: *
User-agent: WebGrab+Plus
So nothing expire
EDIT:
Yes no cookie....just robot file
That is the robots.txt file, not cookies. Which I have done. If you want to grab listings from a particular area, using a local channels file you need to visit the site in a browser to specify the location. That location is stored in a cookie and it is that cookie that is expiring requiring another manual visit to the site. So I need a way of stopping the cookie expiring, or a way to programatically visit the site to reset it. I could use selenium but that seems a bit over the top.
there is no cookie file needed.
creating a channel list is a 3 step process.
1. create a list for your selected zip code,this will create a privider list for the zip code you choosed.
2.using the channel line from step 1 you select the provider you want.'
3. using the channel line from step 2 u create a actual channel list for the zip/provider u picked.
least this is how i do it and it works flawlesssly.
how the ini your using does it i dont know.
doing it the way i said above u can create and use a channel list from any provider and any zip code in the same ini file.
the problem is ini is encrypted so u cannot see the separate sections(they wont exists anyway) as update mode c runs the first one.
creators need to figure this out,i already complained about this problem long ago.
cookies wont do chit as they aint even used,each step add more info so in the last step the channel list is created for the zip/ provider you selected.
in short..
step 1 save your zip code u selected.
step 2 save the zip and provider id u selected.
step 3 creates the channel xml for both.
there is no cookie that will do this as its not even used during the process.
I think there is a bit confusion.
The tvtv.su.ini is not encrypted, see https://github.com/SilentButeo2/webgrabplus-siteinipack/blob/master/site...
then follow Blackbear instructions :
1. On the website https://www.tvtv.us/ select zip code i.e. 10003
2. go to provider webpage https://www.tvtv.us/ny/cooper/10003 and select provider you want i.e. Dish Network - National, NY
3. you will land on webpage https://www.tvtv.us/ny/cooper/10003/lu2384D
4. use the last numeric part after lu (as indicated in siteini channel creation) channel update="i" site="tvtv.us" site_id="2384D" xmltv_id="tvtv New York">tvtv New York (add complete line with tags.
Note:
You can search providers by city name or zip code....no difference as you will only need the last part of the link after "lu"
so for https://www.tvtv.us/ny/new-york-mills/lu139162 will be
site_id="139162"
Again is all explained in siteini channel creation block. Then you need only the robot part, no cookie.
Thanks mat8861, I had already done that and it works fine for a few days, but then it stops working. If I go manually to tvtv.us in chrome it asks me for the location again. Once I have entered it, the grabbing starts working. Hence the reason I think it is a cookie problem. I'm assuming the location is stored in a cookie. If the cookie has expired any visit to tvtv.us will ask for the location, which is confusing the grabber as it is presented with the location request page instead of the actual listings.
Interestingly I have installed wg++ on a Raspberry Pi, copied over the config and tvtv channel list I created and run it without ever visiting the tvtv.us website and it seems to be running fine. Must be something to do with the setup on my windows PC.
Forget about cookies. As in raspberry, linux or windows the only thing needed is give read only to tvtv.us.robot file (Another trick discovered by BlackBear199)