Provided by: libxml-rss-simplegen-perl_11.11-6_all 
NAME
XML::RSS::SimpleGen - for writing RSS files
SYNOPSIS
# A complete screen-scraper and RSS generator here:
use strict;
use XML::RSS::SimpleGen;
my $url = q<http://www.exile.ru/>;
rss_new( $url, "eXile", "Moscow-based Alternative Newspaper" );
rss_language( 'en' );
rss_webmaster( 'xxxxx@yourdomain.com' );
rss_twice_daily();
get_url( $url );
while(
m{<h4>\s*<a href='/(.*?)'.*?>(.*?)</a>\s*</h4>\s*<p.*?>(.*?)<a href='/}sg
) {
rss_item("$url$1", $2, $3);
}
die "No items in this content?! {{\n$_\n}}\nAborting"
unless rss_item_count();
rss_save( 'exile.rss', 45 );
exit;
DESCRIPTION
This module is for writing RSS files, simply. It transparently handles all the unpleasant details of RSS,
like proper XML escaping, and also has a good number of Do-What-I-Mean features, like not changing the
modtime on a written-out RSS file if the file content hasn't changed, and like automatically removing any
HTML tags from content you might pass in.
This module isn't meant to have the full expressive power of RSS; instead, it provides functions that are
most commonly needed by RSS-writing programs.
INTERFACE
This module provides a bunch of functions for starting an RSS feed in memory, putting items into it, and
saving it to disk (or printing it as a string, as in a CGI). If you prefer an object-oriented interface
(obviously more useful if you're composing several feeds at once), then you can use this module as a
class whose methods are the same as the function names minus "rss_". Except for this detail of the
naming, the functions and methods are the same, behave the same, and take the same arguments.
That is, this functional code:
use XML::RSS::SimpleGen;
my $url = q<http://www.exile.ru/>;
rss_new( $url, "eXile" );
rss_language( 'en' );
get_url( $url );
...
does the same work as this OO code:
use XML::RSS::SimpleGen ();
my $url = q<http://www.exile.ru/>;
my $rss = XML::RSS::SimpleGen->new( $url, "eXile");
$rss->language( 'en' );
$rss->get_url( $url );
...
(Note that the function "get_url" doesn't have a leading "rss_", so its method name is the same as its
function name. It's the one exception.)
If this talk of objects puzzles you, see HTML::Tree::AboutObjects in the "HTML-Tree" dist, and/or see the
chapter "User's View of Object-Oriented Modules" in my book Perl & LWP
(<http://www.amazon.com/exec/obidos/ASIN/0596001789>). (The book is also useful as an extended
discussion of screen-scraping.)
Note: in the code below, I use the word "accessor" a lot, to refer to a function or method that you can
call two possible ways: 1) like "foo(val)" to set the "foo" attribute to the value val, or 2) like
"foo()" to return the value of the "foo" attribute.
FUNCTIONS
"rss_new( url );"
"rss_new( url, title );"
"rss_new( url, title, description );"
or: "$rss = XML::RSS::SimpleGen->new(...);"
This function creates a new RSS feed in memory. This should be the first "rss_whatever" function you
call in your program. If you call it again, it erases the current object (if any) and sets up a new
one according to whatever parameters you pass.
The parameters are the full URL, the title, and the description of the site (or page) that you're
providing an RSS feed of. The description is optional, but you should provide at least a URL and
title.
Examples:
rss_new( $url, "eXile", "Moscow-based Alternative Newspaper" );
rss_new( 'http://www.mybazouki.com/news/', "Bazouki News!" );
(As a method, XML::RSS::SimpleGen->new simply returns a new RSS object.)
the accessor "rss_language(language_tag)"
This declares what language this RSS feed is in. It must be an RFC3066-style language tags like
"en", or "en-US", or "zh-TW". (See I18N::LangTags::List for a list.) If you don't set the feed's
language, it defaults to "en", for generic English.
If you call this function without a parameter, it returns the current value of the RSS feed's
language. For example:
print "I'm making an RSS feed for ", rss_language(), "!\n";
The same is true for all the functions that I label as "accessors".
the accessor "rss_item_limit(number)"
This sets the maximum number of items that this feed will show.
The default value is 0, meaning that there is no maximum.
If you set it to a positive number N, then the feed will show only the first N items that you declare
with "rss_item". (Or, if you set "rss_history_file", then the newest N items that you declare with
"rss_item".)
If you set it to a negative number -N, then the feed will show only the last N items that you declare
with "rss_item". (Or, if you set "rss_history_file", then the oldest N items you declare with
"rss_item", which is unlikely to be useful!)
the accessor "rss_webMaster(email-address)"
This declares what email address you, the RSS generator manager, can be reached at. Example:
rss_webMaster( 'sburke@bazouki-news.int' );
"rss_history_file( filename )"
This declares that you want this RSS feed to keep track of what items are new, and to list them first
when the RSS is emitted. To do this, the RSS generator has to store information in a file, where it
tracks its "history", i.e., when was the first time it saw given URLs, and the most recent time it
saw given URLs.
Typical usage is:
rss_history_file( 'thisrssfeed.dat' );
You should call "rss_history_file" before you make any calls to "rss_item".
The history-file feature is meant for cases where your RSS-generator program calls "rss_item" on
every link it sees, but only wants the new links to appear in the RSS output. (This can be a good
approach if you're making an RSS feed of a page like "http://www.guardian.co.uk/" where there's some
new links (to the recently added stories), but also links to some days-old stories, and also links to
some always-there things like "Archive Search" and "Contact Us" pages.
Once you call rss_history_file, the specified file is read in. The in-memory history (stored in the
RSS object) is updated as you call "rss_item". But the file isn't updated until you call rss_save.
(A do-what-I-mean side effect of calling "rss_history_file" is that it sets rss_item_limit to 25 if
it is currently 0.)
(Incidentally, if you're using rss_history_file as part of a CGI that emits RSS data, instead of a
program that just saves to an RSS file, then things will get complicated. You'll need to call an
internal method to explicitly commit the history file to disk, and you'll need a semaphore file to
avoid race conditions. Email me for full info.)
"rss_item( url );"
"rss_item( url, title );"
"rss_item( url, title, description );"
This adds a new item to the current feed. You will need to specify the URL to add (and it should be a
valid-looking URL, starting with "something:", and not containing any spaces). You may also specify
the title, but it's optional. And finally, you can optionally specify a description. (You can
remember this because it starts with the essential item first, and progresses toward the most
optional.)
Leading and tailing whitespace is removed from whichever of url, title, and description are defined
values, and HTML is parsed out.
A simple usage:
rss_item(
"http://www.harpers.org/MostRecentWR.html",
"Harper's Magazine's Weekly Review"
);
Although in practice, a typical call won't have string constants, but will instead be like the
example in the Synopsis sectios, namely:
rss_item("$url$1", $2, $3);
Incidentally, as a do-what-I-mean feature, if the first parameter doesn't look like a URL but one of
the others does, then this error is silently forgiven. This is so you can occasionally slip up and
forget the order of the parameters.
(In the unlikely event where you need to avoid the HTML-removal features, you can do this by passing
scalar-references instead of normal strings, like so: "rss_item($url, $title, \$not_to_be_escaped)".)
"rss_item_count()"
This returns the number of items you've declared. I anticipate that its main usage will be something
like:
die "What, no objects found at $url ?!"
unless rss_item_count();
or, maybe...
exit unless rss_item_count();
...depending on how/whether you'd want to react to cases where you don't see anything to put into an
RSS feed.
Note that the parens are optional, since this command takes no options (just like Perl's "time()"
function).
"rss_image( url, h, w );"
This declares that you want to declare a particular image as the logo for this feed. Most feeds
don't have such a thing, and most readers just ignore it anyway, but if you want to declare it, this
function is how. The three parameters, which are all required, are: the image's URL, its height in
pixels, and its width in pixels. According to various specs, the width should/must be between 1 and
144, an the height should/must be between 1 and 400.
A typical usage:
rss_image("http://interglacial.com/rss/weebl.gif", 106, 140);
Be careful not to mix up the height and width.
"rss_save( filename );"
"rss_save( filename, max_age_days );"
This saves the RSS date to the file you specify. If the RSS data hasn't changed, the file (and its
modtime) aren't altered. The optional max_age_days parameter means that if ever the file exists, and
its content hasn't changed for that many days or longer, then the program should die with a warning
message. For example, in the case of a screen-scraper for a site that we know should (in theory)
change its content at least weekly, we might save the RSS file with:
rss_save("whatever.rss", 17);
# Scream if the feed is unchanged for 17 days.
The seventeen there is gotten by assuming that just maybe the site might skip two weeks for a
vacation now and then, and might even put out the pre-vacation issue a few days early -- but that if
ever the program notices that the data hasn't changed for 17 days, then it should emit error
messages. If you want to disable this feature on a one-time basis, just change the modtime (like via
"touch") on the whatever.rss file.
If you don't specify a "max_age_days" value, then this whole complain-if-it's-old feature is
disabled.
"rss_as_string();"
This returns the RSS-XML data as a string. This function is called internally by the rss_save
function; but you might want to call it explicitly, as in a CGI, where your CGI would probably end
like this:
print "Content-type: application/xml\n\n", rss_as_string();
exit;
"get_url( url );"
"$content = get_url( url );"
or: "$content = $rss->get_url(...);"
or: "$content->get_url(...);"
This tries to get the content of the given url, and returns it.
This is quite like LWP::Simple's "get" function, but with some additional features:
• If it can't get the URL's content at first, it will sleep for a few seconds and try again, up to
about five times. (This is to avoid the case of the URL being temporarily inaccessible simply
because the DNS is a bit slow, or because the server is too busy.)
• If it can't get the content, even after several retries, it will abort the program (like a
"die"). If you want to override this behavior, then call it as "eval { get_url($url) };"
• If you call the function in void context (i.e., not using its return value), then the function
assigns the URL's content to $_. That's so you can write nice concise code like this:
get_url $thatsite;
m/Top Stories Tonight/ or die "What, no top stories?";
while( m{<a class="top" href="(.*?)">(.*?)</a>}g ) {
rss_item("$thatsite/$1", $2);
}
• This returns the content of the URL not exactly as-is, but after changing its newlines to native
format. That is, if the contents of the URL use CR-LF pairs to express newlines, then "get_url"
changes these to "\n"'s before returning the content. (Similarly for old MacOS newline format.)
Clearly this is wrong in you're dealing with binary data; in that case, use LWP::Simple's "get"
directly.
• Finally, as a resource-conversation measure, this function will also try to call "sleep" a few
times if it sees several quick calls to itself coming from a program that seems to be running
under crontab. As most of my RSS-generators are crontabbed, I find it very useful that I can
have however many "get_url"'s in my crontabbed programs without worrying that they'll take even a
noticeable part of the server's bandwidth.
"rss_hourly" or "rss_daily" or "rss_twice_daily" or "rss_thrice_daily" or "rss_weekly" or
"rss_every_other_hour"
Calling one of these functions declares that this feed is usually generated at the same time(s) every
day (or every week, in the case of "rss_weekly"). And, where it's not just once a day/week, these
multiple times a day are evenly spaced. These functions then set the feed's "updatePeriod",
"updateBase", "updateFrequency", "skipHours", "skipDays", and "ttl" elements appropriately, so that
RSS readers can know at at what times there could (or couldn't) be new content in this feed.
In other words: use "rss_twice_daily" if this feed is updated at about the same time every day and
then again 12 hours later. Use "rss_thrice_daily" if this feed is updated at the same time daily,
and then 8 hours later, and then 8 hours later. And use "rss_every_other_hour" if the feed updates
at about n minutes past every even numbered hour, or every odd-numbered hour.
Clearly I mean these functions to be used in programs that are crontabbed to run at particular
intervals, as with a crontab line like one of these:
52 * * * * ~/thingy # => rss_hourly
52 23 * * * ~/thingy # => rss_daily
52 4,16 * * * ~/thingy # => rss_twice_daily
52 5,13,21 * * * ~/thingy # => rss_thrice_daily
52 23 * * 3 ~/thingy # => rss_weekly
52 */2 * * * ~/thingy # => rss_every_other_hour
Clearly there aren't "rss_interval" functions for all the scheduling possibilities programs -- if you
have a program that has to run at 6am, 8am, 1pm, and 4pm, there's no function for that. However, the
above crontab lines (or with minor changes, like "1,9,17" instead of "5,13,21") are just fine for
almost every RSS feed I've run.
An aside: I recommend running the programs at about 52 minutes past the hour, generally in series,
like so:
52 5,13,21 * * * ~/thingy ; ~/dodad ; ~/makething ; ~/gizmo
However, your mileage may vary.
Incidentally, these functions take no arguments, so the parentheses are optional. That is, these two
lines do the same thing:
rss_hourly;
rss_hourly();
MINOR FUNCTIONS
These are functions that you probably won't need often, or at all. I include these for the sake of
completeness, and so that advanced users might find them useful in some cases.
"rss_skipHours( gmt_hour_num, gmt_hour_num, ... );"
This function directly sets the "skipHours" element's values to the specified GMT hour numbers.
"rss_updateHours();"
"rss_updateHours( gmt_hour_num, gmt_hour_num, ... );"
This function is a wrapper around "rss_skipHours" -- you call "rss_updateHours" with a list of GMT
hour numbers, and "rss_updateHours" will call "rss_skipHours(0 .. 23)" except without whatever hour
numbers you specified.
If you call with an empty list (i.e., "rss_updateHours();"), then we uses "gmtime" to find out the
current hour (and rounds it up if it's after 50 minutes past), basically just as if you'd called:
rss_updateHours( (gmtime(600+time()))[2] );
"rss_skipDays();"
"rss_skipDays( gmt_day_num, gmt_day_num, ... );"
"rss_skipDays( gmt_day_name, gmt_day_name, ... );"
This function directly sets the "skipDays" element's values to the specified weekdays. Note that
this accepts either integers (like 6 for Saturday, Sunday being either 0 or 7), or their exact
English names.
If you use the "skipDays" field, consider that it refers to days figured by GMT, not local time. For
example, if I say to skip Saturdays, that means Saturdays GMT, which in my timezone (Alaska) starts
in the middle of Friday afternoon.
"rss_updateDays();"
"rss_updateDays( gmt_day_num, gmt_day_num, ... );"
"rss_updateDays( gmt_day_name, gmt_day_name, ... );"
This function is a wrapper around "rss_skipDays" -- you call "rss_updateDays" with a list of GMT day
names/numbers, and "rss_updateDays" will call "rss_skipDays(0 .. 6)" except without whatever days you
specified.
If you call with an empty list (i.e., "rss_updateDays();"), then we uses "gmtime" to find out the
current day (GMT!), basically just as if you'd called:
rss_updateDays( (gmtime(600+time()))[6] );
"rss_updatePeriod( periodstring );"
This function directly sets the "sy:updatePeriod" element's value to the period specified. You must
specify one of the strings: "yearly", "monthly", "weekly", "daily", "hourly". I advise using
"weekly" only if you know what you're doing, and "yearly", "monthly" only if you really know what
you're doing.
"rss_updatePeriod( periodstring, int, base );"
This is a shortcut for "rss_updatePeriod(periodstring); rss_updateFrequency(int)"
"rss_updatePeriod( periodstring, int, base );"
This is a shortcut for "rss_updatePeriod(periodstring); rss_updateFrequency(int);
rss_updateBase(base)"
"rss_updateBase( iso_date_string );"
"rss_updateBase( epoch_time );"
This function directly sets the "sy:updateBase" element's value to the moment specified. If you pass
in an epoch time, it is converted to an ISO date string.
the accessor "rss_updateFrequency( integer );"
This function directly sets the "sy:updateFrequency" element's value to the value specified. The
value has to be a nonzero positive integer.
For example, this means that this feed updates at/by the start of every hour and 30 minutes past:
rss_updateBase('2000-01-01T00:00-00:00');
rss_updateFrequency(2);
rss_updatePeriod('hourly'); # 2*hourly means "twice an hour"
Recall that this can also be done with the the "rss_updatePeriod( per, freq, base )" shortcut, like
so:
rss_updateBase('hourly', 2, '2000-01-01T00:00-00:00');
the accessor "rss_retention(number)"
If you are using an "rss_history_file(file)", the history file will accrete a list of all URLs it has
seen. But to keep this file from potentially getting immense, items that haven't been seen for a
while are thrown out. The period of time a feed's items go unseen before each is forgotten is called
that feed's retention, and is expressed in seconds.
The default retention value is 32 days (i.e., 32*24*60*60, the number of seconds in 32 days). If you
wanted to change it to just a week, you would do this with "rss_retention(7*24*60*60)".
As a special case, a zero or negative value for the retention means to never clear anything from the
history file, no matter how long it has gone unseen.
"rss_add_comment( strings );"
Call this function if you want to add extra XML comments to this RSS file. For example, if you call
this:
rss_add_comment(
"Our terms of use: http://wherever.int/rsstou.html",
"Any questions? Ask jimmy@wherever.int",
);
...then this RSS feed will contain this XML fairly early on in the file:
<!-- Our terms of use: http://wherever.int/rsstou.html -->
<!-- Any questions? Ask jimmy@wherever.int -->
the accessor "rss_css( url )"
This defines the given URL as being the XML-CSS stylesheet for this RSS feed. The default value is
"./rss.css" if "-e "rss.css"" is true, otherwise is the value http://www.interglacial.com/rss/rss.css
the accessor "rss_xsl( url )"
This defines the given URL as being the XML-XSL stylesheet for this RSS feed. The default value is
none.
The accessors "rss_url( string ), rss_title( string ), rss_description( string )"
These define this feed's URL, title, and description. These functions are just for completeness,
since it's simpler to just specify any/all of these parameters in the call to "rss_new".
the accessor "rss_ttl( number )"
This sets the parameter of this RSS feed's "ttl" element, which suggests how long (in minutes, not
seconds!) an RSS reader should wait after it polls a feed until it polls it again. For example,
"rss_ttl(90)" would suggest that a reader should not poll this feed more often than every 90 minutes.
(This element is somewhat obsolescent next to the newer and more informative "sy:update*" elements,
but is included for backward compatibility.)
the accessor "rss_allow_duplicates( boolean )"
This controls whether or not duplicate items are filtered out out the feed. By default this is on.
Note that duplicates are detected only by their URL, so if you call this:
rss_item('http://foo.int/donate', "Give!");
rss_item('http://foo.int/donate', "We need money!");
rss_save('begging.rss');
...then only the first will appear in the feed, since the second item has a URL that is already being
saved in this feed. (However, "rss_item_count" is still 2, because filtering out duplicates is
something that only happens as the feed is saved.)
the accessor "rss_docs( url )"
This sets the value of the not-generally-useful "doc" RSS element. The default value is
"./about_rss.html" if "-e "about_rss.html"" is true, otherwise
"http://www.interglacial.com/rss/about.html".
the accessors "rss_image_url(url), rss_image_width(number), rss_image_height(number),
rss_image_title(text), rss_image_link(url), rss_image_description(text)"
These are for manually setting the values of this feed's image element's subelements:
<image>
<url> (rss_image_url) </url>
<width> (rss_image_width) </width>
<height> (rss_image_height) </height>
<title> (rss_image_title) </title>
<link> (rss_image_link) </link>
<description> (rss_image_description) </description>
</image>
You rarely need to call any of these "rss_image_whatever" functions -- usually just calling
"rss_image( url, h, w );" is enough.
RSS VERSION
RSS feeds emitted by this module are basically according to v0.92 RSS, with a very few extensions from
v2.0 RSS. They are not RDF files.
SEE ALSO
XML::RSS
<http://my.netscape.com/publish/formats/rss-0.91.dtd>
<http://blogs.law.harvard.edu/tech/rss>
<http://directory.google.com/Top/Reference/Libraries/Library_and_Information_Science/Technical_Services/Cataloguing/Metadata/RDF/Applications/RSS/Specifications/>
<http://feedvalidator.org/>
You might also like my book Perl and LWP, which discusses the many screen-scraping techniques that you
would use for extracting data from HTML to make into RSS feeds:
<http://www.oreilly.com/catalog/perllwp/>
<http://www.amazon.com/exec/obidos/ASIN/0596001789/>
<http://www.amazon.co.uk/exec/obidos/ASIN/0596001789/t>
<http://interglacial.com/d/scrapers> -- examples of Perl programs that produce RSS's (which are visible
at <http://interglacial.com/rss/> )
COPYRIGHT AND DISCLAIMERS
Copyright (c) 2003,4 Sean M. Burke. All rights reserved.
This library is free software; you can redistribute it and/or modify it under the same terms as Perl
itself.
This program is distributed in the hope that it will be useful, but without any warranty; without even
the implied warranty of merchantability or fitness for a particular purpose.
Portions of the data tables in this module are derived from the entity declarations in the W3C XHTML
specification.
Currently (January 2004), that's these three:
http://www.w3.org/TR/xhtml1/DTD/xhtml-lat1.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-special.ent
http://www.w3.org/TR/xhtml1/DTD/xhtml-symbol.ent
Portions of the code in this module were adapted from parts of Gisle Aas's LWP::Simple and the old (v2.x)
version of his HTML::Parser.
AUTHOR
Sean M. Burke "sburke@cpan.org"
perl v5.26.2 2018-05-06 XML::RSS::SimpleGen(3pm)