aboutsummaryrefslogtreecommitdiff
path: root/IkiWiki/Plugin/aggregate.pm
Commit message (Collapse)AuthorAge
* useragent: Automatically choose whether to use LWPx::ParanoidAgentSimon McVittie2019-02-26
| | | | | | | | | | | | | The simple implementation of this, which I'd prefer to use, would be: if we can import LWPx::ParanoidAgent, use it; otherwise, use LWP::UserAgent. However, aggregate has historically worked with proxies, and LWPx::ParanoidAgent quite reasonably refuses to work with proxies (because it can't know whether those proxies are going to do the same filtering that LWPx::ParanoidAgent would). Signed-off-by: Simon McVittie <smcv@debian.org>
* aggregate: Fix aggregation of posts without a titleJoey Hess2018-10-28
| | | | | | | | | | | | | | | | When an aggregated post lacked a title, the code first prepended the $feed->{dir} to it, and only then checked if it had zero length. So, that check could never succeed and it was possible to end up with $page="dir/", and writing to that would of course fail. (Same problem could also occur when the whole title got sanitized away by the wiki_file_regexp.) Fixed by simply checking earlier if $page is empty. Based on a patch by Alexandre Oliva which got lost in a maze of email folders all alike for over two years despite him mentioning it to me at least once in person.
* Silence "used only once: possible typo" warnings for variables that are part ↵Simon McVittie2016-01-19
| | | | of modules' APIs
* protect $@ whenever a block using $@ is non-trivialSimon McVittie2014-02-21
| | | | | | | | | | | | | | | | | | | | | | | | As noted in the Try::Tiny man page, eval/$@ can be quite awkward in corner cases, because $@ has the same properties and problems as C's errno. While writing a regression test for definetemplate in which it couldn't find an appropriate template, I received <span class="error">Error: failed to process template <span class="createlink">deftmpl</span> </span> instead of the intended <span class="error">Error: failed to process template <span class="createlink">deftmpl</span> template deftmpl not found</span> which turned out to be because the "catch"-analogous block called gettext before it used $@, and gettext can call define_gettext, which uses eval. This commit alters all current "catch"-like blocks that use $@, except those that just do trivial things with $@ (string interpolation, string concatenation) and call a function (die, error, print, etc.)
* aggregrate: Fix several bugs in handling of empty and colliding titles when ↵Joey Hess2013-12-28
| | | | generating filenames.
* cookiejar is now a core config setting.Amitai Schlair2013-08-03
|
* Honor proxy env vars and reliably honor cookiejar.Amitai Schlair2013-07-27
|
* Show author in addition to feedname, if different.Amitai Schlair2013-07-17
| | | | | | While here, mollify http://validator.w3.org/feed/ and s/dcterms:creator/dc:creator/g, which happens to make rss2email see and do nice things with authors.
* aggregate: When run with --aggregate, if an aggregation is already running, ↵Joey Hess2012-12-27
| | | | | | | don't go on and --refresh. This way, if a previous aggregation job is running, we don't add additional load doing work that job will do anyway.
* prune: do not prune beyond an optional base directory, and add a testSimon McVittie2012-04-07
| | | | | | | | Previously, prune("wiki/srcdir/sandbox/test.mdwn") could delete srcdir or even wiki, if they happened to be empty. This is rarely what you want: there's usually some base directory (destdir, srcdir, transientdir or another subdirectory of wikistatedir) beyond which you do not want to delete.
* better long filename detection methodJoey Hess2011-06-10
| | | | Let's just try to write and fall back to a short ugly filename on error.
* aggregate: Improve checking for too long aggregated filenames.Joey Hess2011-06-10
| | | | | | | | Two problems fixed: 1. Files are written with a .ikiwiki-new suffix, which has to be taken into account. 2. Need to count length of bytes, not of unicode characters.
* aggregate, pinger: Use Net::INET6Glue if available to support making ipv6 ↵Joey Hess2011-05-09
| | | | | | | connections. Making outgoing ipv6 connections for openid auth is still broken; the glue module does not seem to solve that, so I did not make openid use it.
* fix var nameJoey Hess2011-03-21
|
* comment about lockingJoey Hess2011-03-21
|
* call aggregate checkconfig lastJoey Hess2011-03-21
| | | | | | | Since the plugin abuses the checkconfig hook to launch aggregation when in --aggregate mode, it should give other plugins that have checkconfig hooks a chance to run before they are possibly used in rendering the aggregated content.
* aggregate: Read cookies from ~/.ikiwiki/cookies by default. Also, the ↵Joey Hess2011-03-21
| | | | cookiejar configuration setting can be used by other plugins to provide a custom `cookie_jar` object for LWP::UserAgent. (Thanks, schmonz)
* minor optimisationJoey Hess2011-02-09
| | | | | | Assume the aggregated content is only going to be in one of the directories, and so stop if it's successfully removed from the transientdir.
* Put newly aggregated pages in the transient underlaySimon McVittie2011-01-07
|
* factored out an urlabs from aggregate and cgiJoey Hess2011-01-05
|
* avoid fatal error if aggregate page template could not be foundJoey Hess2010-09-27
| | | | That template is user-controlled.
* needsbuild hook interface changed; the hooks should now return the modified ↵Joey Hess2010-09-07
| | | | array of things that need built. (Backwards compatability code keeps plugins using the old interface working.)
* revert bugfixJoey Hess2010-07-06
| | | | Not needed; lastupdate will be 0 for new feeds.
* bugfixJoey Hess2010-07-06
|
* aggregate: Write timestamp next aggregation can happen to ↵Joey Hess2010-07-06
| | | | .ikiwiki/aggregatetime, to allow for more sophisticated cron jobs.
* setup file orderingJoey Hess2010-02-12
|
* add section informationJoey Hess2010-02-12
|
* aggregate: Avoid resetting ctime when an item md5 changes.Joey Hess2009-06-12
| | | | | | | | | | | Besides being wrong to do, this could lead to the wrong item being expired, as follows: If B is added and at the same time A is changed, then A's ctime may be set to the current time, while B's is set to its creation time. Thus the new item, A, is incorrectly removed as older. (This interacted especially badly with the bug fixed by 90b4d079605b72bb50d1da41402d994960e10937.)
* aggregate: Fix storing of changed md5.Joey Hess2009-06-12
| | | | | | | The aggregate state merge code neglected to merge changes to the md5 field of an item. Therefore, if an item's md5 changed after initial aggregation, it would be updated, and rewritten, each time thereafter. This was wasteful and indirectly led to some expire problems.
* Work around XML::Atom strangeness that results in double-encoded postsSimon McVittie2009-02-03
| | | | | | | | | See [[bugs/Aggregated_Atom_feeds_are_double-encoded]]. By default, XML::Atom outputs strings of UTF-8 bytes with the Perl UTF8 flag stripped off, which IkiWiki assumes to be Latin-1 and re-encodes as UTF-8 on output. XML::Feed does not currently (0.41-1) set the magic variable to change this behaviour (I've filed a bug on CPAN), but IkiWiki can usefully set the same variable as a workaround.
* finalise version 3.00 of the plugin apiJoey Hess2008-12-23
|
* enable aggregate_internal by defaultJoey Hess2008-12-23
|
* avoid uninitialized valueJoey Hess2008-12-19
| | | | | This can happen when a new field, such as the new lasttry, is added.
* aggregate: If a feed fails to be downloaded, try again immediatly next time ↵Joey Hess2008-12-17
| | | | aggregation is run, even if the usual time has not passed. Closes: #508622 (Michael Gold)
* Coding style change: Remove explcit vim folding markers.Joey Hess2008-12-17
|
* avoid uninitialized value warningJoey Hess2008-11-18
|
* improve escaping of wikilinks and preprocessor directivesJoey Hess2008-11-18
| | | | The old method failed for '[' x 3.
* call decode_utf8 inside evalJoey Hess2008-11-17
| | | | | | | holger reported that decode_utf8 was crashing with perl 5.8.8. Earlier, I thought that passing 0 to the function avoided this with old perls, but that was apparently not enough, it still crashes. So, put it inside the eval, so we can at least recover from it crashing.
* aggregate: Try to query XML::Feed for the base url when derelevatising ↵Joey Hess2008-11-06
| | | | links. Since this needs the just released XML::Feed 0.3, as well as a not yet released XML::RSS, it will fall back to the old method if no xml:base info is available.
* fix relativedate timezone inclusionJoey Hess2008-10-19
| | | | | | The machine parseable date needs to include a timezone. Also, simplified the interface for date display.
* mark up date so relativedate will workJoey Hess2008-10-19
|
* aggregate: Avoid bug that caused immediate expiration of items with a date ↵Joey Hess2008-10-16
| | | | in the future.
* Removed the pagefile function, which was confusingly close in name toJoey Hess2008-09-29
| | | | | | | | newpagefile. Note that newpagefile is not used here (or in recentchanges) because the internal use pages they generate are transient and unlikely to benefit from being put each in their own subdir.
* Export pagetitle, titlepage, linkpage.Joey Hess2008-09-27
|
* typoJoey Hess2008-09-27
|
* aggregate: Avoid uninitialized value warnings for pages with no recorded ctime.Joey Hess2008-09-25
| | | | | | | I saw this in the wild, apparently a page was not present on disk, but was in the aggregate db, and not marked as expired either. Not sure how that happened, but such pages should get marked as expired since they have an effectively zero ctime.
* fix check for nonexistant contentJoey Hess2008-09-17
|
* store ctime for use by expiry codeJoey Hess2008-09-17
| | | | | The expiry code does need to make sure to sort in ctime order, even if expiring by count, so it expires the right ones.
* aggregate: Support atom feeds with only a summary element, and no content ↵Joey Hess2008-09-17
| | | | elements.
* aggregate: Allow expirecount to work on the first pass. (expireage still ↵Joey Hess2008-09-17
| | | | needs to wait for the pages to be rendered though)
generated by cgit v1.2.3 (git 2.25.4) at 2024-05-19 07:09:16 +0000