thh/newsstats
1
0
Fork 0
Code Issues 3 Pull requests Projects Releases Packages Activity

Update documentation.

Browse source 
- Fix clientstats doc (copied from hoststats).
- Add some more examples ro README.

Signed-off-by: Thomas Hochstein <thh@thh.name>
This commit is contained in:
Thomas Hochstein 2025-06-01 16:39:25 +02:00
parent 0b87e81b08
commit 66890b68d8
11 changed files with 329 additions and 263 deletions
Download patch file Download diff file Expand all files Collapse all files

1
doc/ChangeLog
View file

@ -16,6 +16,7 @@ NewsStats 0.4.0 (unreleased)
* Add ClientStats to dopostingstats.
* Let dopostingstats default to last month.
* Set executable bit for new scripts.
* Update documentation.
NewsStats 0.3.0 (2025-05-18)
* Extract GroupStats (in gatherstats) to subroutine.

55
doc/INSTALL
View file

@ -1,11 +1,12 @@
NewsStats (c) 2010-2013, 2025 Thomas Hochstein <thh@thh.name>
NewsStats is a software package used to gather statistical information
from a live Usenet feed and for its subsequent examination.
NewsStats is a software package that can be used to collect
statistical information from a live Usenet feed and then analyze it
to create statistical reports.
This script package is free software; you can redistribute it and/or
modify it under the terms of the GNU Public License as published by
the Free Software Foundation.
This package is free software; you can redistribute it and/or modify
it under the terms of the GNU Public License as published by the Free
Software Foundation.
---------------------------------------------------------------------
@ -16,9 +17,10 @@ INSTALLATION INSTRUCTIONS
* Download the current version of NewsStats from
<https://th-h.de/net/software/newsstats/>.
* Untar it into a directory of your choice:
* Untar it into a directory of your choice, i.e. /srv/newsstats:
# tar -xzf newsstats-nn.tar.gz
$ cd /srv
$ tar -xzf newsstats-n.n.n.tar.gz
Scripts in this path - at least feedlog.pl - should be executable by the
news user.
@ -28,8 +30,8 @@ INSTALLATION INSTRUCTIONS
* Copy the sample configuration file newsstats.conf.sample to
newsstats.conf and modify it for your purposes:
# cp etc/newsstats.conf.sample etc/newsstats.conf
# vim etc/newsstats.conf
$ cp etc/newsstats.conf.sample etc/newsstats.conf
$ vim etc/newsstats.conf
a) Mandatory configuration options
@ -60,6 +62,9 @@ INSTALLATION INSTRUCTIONS
* DBTableHosts = hosts_de
Table holding data on postings per server.
* DBTableClnts = clients_de
Table holding data on postings per client.
b) Optional configuration options
* TLH = de.alt,news.admin
@ -68,19 +73,21 @@ INSTALLATION INSTRUCTIONS
3) Database (mysql) setup
* Setup your database server with a username, password and
database matching the NewsStats configuration (see 2 a).
* Setup your database server with an username, a password and
(optionally) a database matching the NewsStats configuration
(see 2 a).
* Start the database creation script:
# bin/dbcreate.pl
$ bin/dbcreate.pl
It will setup the necessary database tables and display some
information on the next steps.
It will create the database (if not already present), create the
necessary database tables and display some information on the
next steps.
4) Feed (INN) setup
You have to setup an INN feed to feedlog.pl.
You have to set up an INN feed to feedlog.pl.
* Edit your 'newsfeeds' file and insert something like
@ -90,39 +97,39 @@ INSTALLATION INSTRUCTIONS
:Tc,WmtfbsPNH,Ac:/path/to/feedlog.pl
* You should only feed that hierarchy (those hierarchies ...) to
feedlog.pl you'll want to cover with your statistical
examination. It may be a good idea to setup different feeds (to
different databases ...) for different hierarchies.
feedlog.pl that you want to cover with your statistical analysis.
It may be a good idea to setup different feeds (to different
databases ...) for different hierarchies.
* Please double check that your path to feedlog.pl is correct and
feedlog.pl can be executed by the news user
* Check your 'newsfeeds' syntax:
# ctlinnd checkfile
$ ctlinnd checkfile
* Reload 'newsfeeds':
# ctlinnd reload newsfeeds 'Adding newsstats! feed'
$ ctlinnd reload newsfeeds 'Adding newsstats! feed'
* Watch your 'news.notice' and 'errlog' files:
# tail -f /var/log/news/news.notice
$ tail -f /var/log/news/news.notice
...
# tail -f /var/log/news/errlog
$ tail -f /var/log/news/errlog
Everything should be going smoothly now.
* If INN is spewing error messages to 'errlog' or reporting
continous respawns of feedlog.pl to 'news.notice', stop your feed:
# ctlinnd drop 'newsstats!'
$ ctlinnd drop 'newsstats!'
and investigate. 'errlog' may be helpful here.
* You can restart the feed with
# ctlinnd begin 'newsstats!'
$ ctlinnd begin 'newsstats!'
later.

75
doc/README
View file

@ -1,21 +1,21 @@
NewsStats (c) 2010-2013, 2025 Thomas Hochstein <thh@thh.name>
NewsStats is a software package for gathering statistical data live
from a Usenet feed and subsequent examination.
from a Usenet feed and subsequent analysis.
This script package is free software; you can redistribute it and/or
modify it under the terms of the GNU Public License as published by
the Free Software Foundation.
This package is free software; you can redistribute it and/or modify
it under the terms of the GNU Public License as published by the Free
Software Foundation.
---------------------------------------------------------------------
What's that?
There's a multitude of tools for the statistical examination of
newsgroups: number of postings per month or per person, longest
threads, and so on (see <https://th-h.de/net/usenet/stats/>
[German language] for an incomplete list). Most of them use a per-
newsgroup approach while NewsStats is hierarchy oriented.
There's a multitude of tools to create statistics about newsgroup
usage: number of postings per month or per person, longest threads,
and so on (see <https://th-h.de/net/usenet/stats/> [German language]
for an incomplete list). Most of them use a per-newsgroup approach
while NewsStats is hierarchy oriented.
NewsStats will accumulate data from a live INN feed, allowing you
to process the saved information later on.
@ -40,7 +40,9 @@ Prerequisites
* Perl 5.8.x with standard modules
- Cwd
- Encode
- File::Basename
- Getopt::Long
- Sys::Syslog
* Perl modules from CPAN
@ -50,7 +52,7 @@ Prerequisites
* mysql 5.0.x
* working installation of INN
* a working installation of INN
Installation instructions
@ -67,15 +69,52 @@ Getting Started
table. See the feedlog.pl man page for more information.
You can process that data via 'gatherstats.pl'; currently the
tabulation of postings per group and injection server per month is
supported. Tabulation of clients (newsreaders) is planned. See
the gatherstats.pl man page for more information.
tabulation of postings per group, injection server and posting
agent (newsreader) per month is supported. See the gatherstats.pl
man page for more information.
Example:
bin/gatherstats.pl
will parse raw data from the last month and save the results in
tables for postings per group, server and client, respectively.
Report generation is handled by specialised scripts for each
report type. Currently reports on the number of postings per group
and month and injection server and month are supported; you can
use 'groupstats.pl' and 'hoststats.pl' for that. See the
groupstats.pl and hoststats.pl man pages for more information.
report type: 'groupstats.pl' for postings per group
(s), 'hoststats.pl' for postings per injection server
(s) and 'clientstats.pl' for postings per posting agent. See the
groupstats.pl, hoststats.pl and clientstats.pl man pages for more
information.
Example:
bin/groupstats.pl -o postings-desc
bin/hoststats.pl -o postings-desc
bin/clientstats.pl -o postings-desc -v
will show reports for postings per group, per injection server and
per client (with detailed client versions) for the last month,
using the result tables filled by gatherstats.
To post those reports to Usenet, change postingstats.pl according
to your needs (sender, newsgroups and other headers, translation
of table headers and text templates) and display a test posting
by piping report data into postingstats.pl:
bin/groupstats.pl --nocomments -s -f dump | bin/postingstats.pl
If the result is to your liking, add a pipe to a inews
implementation.
Example:
bin/groupstats.pl --nocomments -s -f dump | bin/postingstats.pl | contrib/tinews.pl -X
More information
See the man pages for 'gatherstats' and the report generating
scripts.
Reporting Bugs
@ -87,7 +126,7 @@ Reporting Bugs
Development
This program is maintained using the Git version control system at
This package is maintained using the Git version control system at
<https://code.virtcomm.de/thh/newsstats/>.
Related projects

22
doc/TODO
View file

@ -1,12 +1,10 @@
NewsStats To-Do List
====================
This is a list of planned bug fixes, improvements and enhancements for
This is a list of possible bug fixes, improvements and enhancements for
NewsStats.
* General
- Improve Documentation
The documentation is rather sparse and could use some improvement.
- Add a test suite
There is currently no kind of test suite or regression tests. Something like
that is badly needed.
@ -27,8 +25,6 @@ NewsStats.
for late creation and deletion), optionally including the previously
mentioned information; and you should be able to get the history of any
group.
- Add other reports
NewsStats should include some other kinds of reports (stats on used clients)
- Add tools for database management
NewsStats should offer tools e.g. to inject postings into the 'raw' database,
or to split databases.
@ -53,23 +49,11 @@ NewsStats.
Some other tests - working database connection, valid database and table
names - would be nice.
+ install/install.pl
- Read current version from a file dropped and updated by installer
- Add / enhance / test error handling
- General tests and optimisations
+ feedlog.pl
- Add / enhance / test error handling
- General tests and optimisations
+ gatherstats.pl
- Use hierarchy information (see GroupInfo above)
- Add gathering of other stats (clients, ...)
- better modularisation (code reuse for other reports!)
- Add / enhance / test error handling
- General tests and optimisations
+ groupstats.pl
- better modularisation (code reuse for other reports!)
+ groupstats.pl, hoststats.pl, clientstats.pl
- better modularisation (code reuse)
- Add / enhance / test error handling
- General tests and optimisations