**------------------------------------------------------------------------------------------------ * @header_start * WebGrab+Plus ini for grabbing EPG data from TvGuide websites * @Site: tvguide.com * @MinSWversion: V1.1.1/56.27 * none * @Revision 14 - [18/03/2017] Netuddki * - Added option for Onscreen OR xmltv_ns Episode System * - Fixed Showicons * @Revision 13 - [24/07/2016] Blackbear199 * - remove index_start/stop modify * @Revision 12 - [22/01/2015] francis De Paemeleere * - adjust to changes of the site * @Revision 11 - [16/01/2015] francis De Paemeleere * - quick fix (index only) * @Revision 10 - [29/06/2014] Jan van Straaten * - added previousshown and repeat * @Revision 9 - [08/06/2014] Ron * - changed () to [] in channel creation (MePo problem) * @Revision 8 - [02/05/2014] Francis De Paemeleere * - adjust to the new timezone implementation of WG++ * @Revision 7 - [05/03/2014] Francis De Paemeleere * - total rewrite of the code * @Remarks: * this is the base version * @header_end **------------------------------------------------------------------------------------------------ site {url=tvguide.com|timezone=UTC|maxdays=16.1|cultureinfo=en-US|charset=UTF-8|titlematchfactor=90|firstshow=2} site {ratingsystem=MPAA} site {episodesystem=onscreen|} *Enable for Onscreen Episode System *site {episodesystem=xmltv_ns|} *Enable for xmltv_ns Episode System urldate.format{datenumber|UNIX|0} scope.range {(urlindex)|end} index_temp_1.modify {substring(type=regex)|'config_site_id' "\(srvID([^)]*)\)"} url_index.modify {replace|##svrID##|'index_temp_1'} index_temp_1.modify {substring(type=regex)|'config_site_id' "SourceId:([^,]*)"} url_index.modify {replace|##SourceId##|'index_temp_1'} index_temp_1.modify {substring(type=regex)|'config_site_id' "Number:([^,]*)"} url_index.modify {replace|##Number##|'index_temp_1'} index_temp_1.modify {calculate(format=D0)|'config_timespan_days' 1 + 1440 *} * calculate how may minutes we want to get (so we only download the days, we want to see) url_index.modify {replace|##Minutes##|'index_temp_1'} end_scope url_index{url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/airings/##svrID##/start/|urldate|/duration/##Minutes##?channelsourceids=##SourceId##%7C##Number##&formattype=json} url_index.headers {customheader=Accept-Encoding=gzip,deflate} index_showsplit.scrub {regex||"ProgramSchedule":.*?}}||} index_start.scrub {regex||"StartTime"\s*:\s*((?:[^\s,])*)\s*,||} index_stop.scrub {regex||"EndTime"\s*:\s*((?:[^\s,])*)\s*,||} index_title.scrub {regex||"Title"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} index_subtitle.scrub {regex||"EpisodeTitle"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} index_temp_3.scrub {regex||"ProgramId"\s*:\s*((?:[^\s,])*)\s*,||} index_temp_4.scrub {regex||"AiringAttrib"\s*:\s*(\d*)||} * binairy: 1= Live, 2=??,4=New, 8=?? * grab season and episode number index_temp_7.scrub {regex||"SeasonNumber":([1-9]+)||} index_temp_7.modify {addstart('index_temp_7' not "")|S} index_temp_8.scrub {regex||"EpisodeNumber":"([1-9]+)||} index_temp_8.modify {addstart('index_temp_8' not "")|E} index_temp_8.modify {addstart('index_temp_7' not "")|'index_temp_7'} index_episode.modify {set(pattern="S'S1'E'E1'")|'index_temp_8'} scope.range {(indexshowdetails)|end} index_title.modify {cleanup(style=jsondecode)} index_temp_9.modify {set|'index_subtitle'} index_temp_9.modify {cleanup(style=regex)} * remove subtitle from title index_title.modify {remove(notnull type=regex)|":\s*'index_temp_9'"} * add flags according to airing attributes index_temp_5.modify {calculate(format=D0)|'index_temp_4' 1 and} index_temp_6.modify {addend('index_temp_5'=="1")| (live)} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 2 and} index_temp_6.modify {addend('index_temp_5'=="2")| (repeat)} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 4 and} index_temp_6.modify {addend('index_temp_5'=="4")| (new)} * next line adds * to new shows, Ttile * = new shows, Title = "not new", 2 places *index_title.modify {addend('index_temp_5'=="4")| *} index_temp_5.modify {calculate(format=D0)|'index_temp_4' 8 and} index_temp_6.modify {addend('index_temp_5'=="8")| (cc)} end_scope index_urlshow.modify {clear} index_urlshow.modify {addstart('index_temp_3' not="")|http://mapi.tvguide.com/listings/details?program='index_temp_3'} index_urlshow.headers {customheader=Accept-Encoding=gzip,deflate} *to speedup the downloading of the detail pages temp_1.scrub {regex||^.*?"seo_url"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} title.scrub {regex||^.*?"title"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} description.scrub {regex||^.*?"description"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} *description.scrub {single||||} rating.scrub {regex||^.*?"rating"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} productiondate.scrub {regex||^.*?"release_year"\s*:\s*([+-]?\d*)||} *productiondate.scrub {regex||^.*?"airdate"\s*:\s*([+-]?\d*)||} *productiondate.modify {calculate(format=utcdate,MM/dd/yyyy)} temp_2.scrub {regex||^.*?"type"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} temp_4.scrub {regex||^.*?"category_id"\s*:\s*([+-]?\d*)||} * genre ID showicon.scrub {regex||"photos":\[[^]]*?\]||} scope.range {(showdetails)|end} title.modify {cleanup(style=jsondecode)} description.modify {cleanup(style=jsondecode)} rating.modify {cleanup(style=jsondecode)} productiondate.modify {cleanup(style=jsondecode)} temp_2.modify {cleanup(style=jsondecode)} *remove subtitle from title title.modify {remove(notnull type=regex)|":\s*'index_temp_9'"} temp_5.modify {calculate(format=D0)|'index_temp_4' 4 and} *next line adds * to new shows, Title * = new shows, Title = "not new", 2 places *title.modify {addend('temp_5'=="4")| *} actor.modify {replace|,|\|} * make actor multi a multi element rating.modify {clear("None")} description.modify {addend|'index_temp_6'} category.modify {addend('temp_4' == "1")|##_##movie} category.modify {addend('temp_4' == "2")|##_##sports} category.modify {addend('temp_4' == "3")|##_##family} category.modify {addend('temp_4' == "4")|##_##news} category.modify {addend|##_##'temp_2'} category.modify {replace|##_##|\|} category.modify {remove|other} * previousshown.modify {substring(type=regex)|'description' "(\(repeat\))"} previousshown.modify {replace(not "")|(repeat)|true} * description.modify {remove(type=regex)|"(\(repeat\))"} *optional removes (repeat) from the description premiere.modify {substring(type=regex)|'description' "(\(new\))"} *premiere.modify {replace(not "")|(new)|true} *description.modify {remove(type=regex)|"(\(new\))"} *optional removes (new) from the description description.modify {cleanup} showicon.modify {substring(type=regex)|"url":"([^"]*)"} *get all the photos for this show showicon.modify {substring(type=element)|2 1} *only get the third image (multi images are not yet supported by WG++) showicon.modify {cleanup(style=jsondecode)} end_scope urlsubdetail.modify {clear} urlsubdetail.modify {addstart('temp_1' not="")|'temp_1'cast} urlsubdetail.modify {clear('temp_1' =="0")} urlsubdetail.modify {cleanup(style=jsondecode)} urlsubdetail.headers{customheader=Accept-Encoding=gzip,deflate} *to speedup the downloading of the detail pages subdetail_temp_1.scrub {regex||(>\s*(?:Active)?\s*<[^>]*>\s*Cast\s*<.*$)||} subdetail_temp_1.modify {remove(type=regex)|(>\s*(?:Former)?\s*<[^>]*>\s*Cast\s*<.*$)} subdetail_actor.modify {substring(type=regex)|'subdetail_temp_1' "tvobject\-cast\-role\-actor[^>]*>([^<]*)"} *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### TIMEZONE FILE CREATION (only to create the xxx-channel.xml file) ** this will create a .channels.xml file (channels by timezone) that will contain all the channels for all the supported timezones ** this can be used to generate a new tvguide.com_channels_byTimezone.xml ** @auto_xml_timezone_start *url_index{url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/schedules/|subpage|/start/|urldate|/duration/1?ChannelFields=Name%2CFullName%2CNumber%2CSourceId&ScheduleFields=&formattype=json&disableChannels=} *subpage.format {list|80001|80002|80003|80004|80005|80006} *index_site_id.scrub {single||||} *scope.range {(channellist)|end} ** ! remark: we loop from the back to the front! *index_temp_4.modify {clear} *index_temp_4.modify {addstart|80006} * *index_site_id.modify {replace|[\{"Channel":|\n(srvIDNNNN)} * "split" the full lising into timezone separate parts * *index_site_channel.modify {substring(type=regex)|'index_site_id' "(srvIDNNNN)\|"FullName":"([^"]*)"[^}]*"SourceId":\d*"} *index_site_id.modify {substring(type=regex)|'index_site_id' "(srvIDNNNN)\|"FullName":"[^"]*"[^}]*"SourceId":(\d*)"} * *index_temp_1.modify {calculate(type=element format=F0)|'index_site_channel' #} *how many elements? Used as loop counter. *index_temp_6.modify {clear} *index_temp_5.modify {clear} *index_temp_6.modify {addstart|'index_site_channel'} *copy original elemets in temp *index_temp_5.modify {addstart|'index_site_id'} *copy original elemets in temp *index_site_channel.modify {clear} *to be refilled later *index_site_id.modify {clear} *to be refilled later *loop {('index_temp_1' > "0" max=5000)|end} *index_temp_1.modify {calculate(format=F0)|1 -} *decrease the loop counter *index_temp_2.modify {substring(type=element)|'index_temp_6' 'index_temp_1' 1} *element to work on *index_temp_3.modify {substring(type=element)|'index_temp_5' 'index_temp_1' 1} *element to work on * *index_site_channel.modify {addstart('index_temp_2' not== "srvIDNNNN")|##_####TIMEZONE##'index_temp_2'} *fill elements again , add a placeholder ##_## as separator *index_site_id.modify {addstart('index_temp_2' not== "srvIDNNNN")|##_####SERVERID##Number:*,SourceId:'index_temp_3'} *fill elements again , add a placeholder ##_## as separator * *index_site_id.modify {replace('index_temp_2' == "srvIDNNNN")|##SERVERID##|(srvID'index_temp_4')} *after all channels are added, change the server id to the correct value *index_site_channel.modify {replace('index_temp_2' == "srvIDNNNN")|##TIMEZONE##|##TIMEZONE'index_temp_4'##} *after all channels are added, change the server id to the correct value *index_temp_4.modify {calculate('index_temp_2' == "srvIDNNNN" format=F0)|'index_temp_4' 1 -} *decrease the server id for the next time (80006, 80005, 80004, ...) * *end_loop *index_site_channel.modify {replace|##TIMEZONE80001##|(Eastern)} *index_site_channel.modify {replace|##TIMEZONE80002##|(Central)} *index_site_channel.modify {replace|##TIMEZONE80003##|(Mountain)} *index_site_channel.modify {replace|##TIMEZONE80004##|(Pacific)} *index_site_channel.modify {replace|##TIMEZONE80005##|(Alaskan)} *index_site_channel.modify {replace|##TIMEZONE80006##|(Hawaiian)} *index_site_channel.modify {replace|##_##|\|} *index_site_id.modify {replace|##_##|\|} *index_site_id.modify {cleanup(removeduplicates=equal,100 link="index_site_channel")} ** replace in channels with [ ] *index_site_channel.modify {replace|)|]} *index_site_channel.modify {replace|(|[} *end_scope ** @auto_xml_timezone_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### PROVIDER FILE CREATION (only to create the xxx-channel.xml file) ** ** @auto_xml_provider_start *url_index{url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/serviceproviders/zipcode/|channel|?formattype=json} *url_index.headers {customheader=Accept-Encoding=gzip,deflate} *to speedup the downloading *index_site_id.scrub {regex||"Id"\s*:\s*([+-]?\d*)||} *index_site_channel.scrub {regex||"Name"\s*:\s*"([^"\\]*(?:\\.[^"\\]*)*)"||} ** @auto_xml_provider_end *** _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ** ##### CHANNEL FILE CREATION (only to create the xxx-channel.xml file) ** ** @auto_xml_channel_start url_index{url|http://mobilelistings.tvguide.com/Listingsweb/ws/rest/schedules/|channel|/start/|urldate|/duration/1?ChannelFields=Name%2CFullName%2CNumber%2CSourceId&ScheduleFields=&formattype=json&disableChannels=} index_site_channel.scrub {regex||"FullName":"([^"]*)"[^}]*"SourceId":\d*||} index_site_id.scrub {regex||"FullName":"[^"]*"[^}]*("Number":"[^"]*"[^}]*"SourceId":\d*)||} scope.range {(channellist)|end} index_site_id.modify {remove(type=regex)|("Sort"\s*:\s*\d*\s*,*)} index_site_id.modify {remove|"} index_site_id.modify {addstart|(srvID'config_site_id')} *add the config site_id, so, we can keep this siteini generic (no need for the user to edit it) index_site_id.modify {cleanup(removeduplicates=equal,100 link="index_site_channel")} replace in channels with [ ] index_site_channel.modify {replace|)|]} index_site_channel.modify {replace|(|[} end_scope ** @auto_xml_channel_end