WendzelNNTPd-OSE Version 2 Documentation

Steffen Wendzel, www.wendzel.de

September 26, 2015


1 Introduction

WendzelNNTPd is a tiny but easy to use Usenet server (NNTP server) for Linux, *nix and BSD. The server is written in C. For security reasons, it is compiled with stack smashing protection by default, if your compiler supports that feature.

1 Features

1 License

WendzelNNTPd uses the GPLv3 license.

2 Database Abstraction Layer

WendzelNNTPd contains a database abstraction layer. Currently supported database systems are SQlite3 and MySQL. New databases can be easily added.

3 Security

WendzelNNTPd contains different security features, the probably most important feature are Access Control Lists (ACLs) as well as the Role Based Access Control (RBAC) system. ACL and RBAC are described in a own chapter. WendzelNNTPd is probably the first Usenet server with support for RBAC.

Another feature are ``invisible newsgroups'': If access control is activate, a user without permission will neither be able to see the existence of a newsgroup, nor will he be able to post to or read the newsgroup, if he would know about its existence.

4 Auto-prevention of double-postings

In case a user sends a posting to two equal newsgroups within one post command's ``Newsgroups:'' header tag, the server will add it only once to save memory on the server and the time of the readers.

5 IPv6

WendzelNNTPd supports IPv6. The server can listen on multiple IP addresses as well as multiple ports.

6 Why this is not a perfect Usenet server

WendzelNNTPd does not implement all NNTP commands, but the (most) important ones. Another problem is that the used regular expression library is not 100% compatible with the NNTP matching in commands like ``XGTITLE''. A 3rd aspect is that WendzelNNTPd cannot share messages with other NNTP servers. Finally, WendzelNNTPd lacks support for encrypted connections.

2 Contribute

See ``CONTRIBUTE'' file in the tarball.

3 History

The project started in 2004 under the name Xyria:cdpNNTPd, as part of the Xyria project that also contained a fast DNS server, called Xyria:DNSd. In 2007, I renamed it to WendzelNNTPd and stopped the development of Xyria:DNSd. Version 1.0.0 was released in 2007, version 2.0.0 in 2011. Since then I primarly fix reported bugs and add minor features to keep the software alive, which is also planned for the next ten years. A detailed history can be found in the ``HISTORY'' file in the tarball.

2 Installation

This chapter provides a guide on how to install WendzelNNTPd-OSE (Open Source Edition) 2.x.[*]

1 Linux/*nix/BSD

To install WendzelNNTPd from source you need to download the provided archive file (e.g. wendzelnntpd-2.0.0.tar.gz) file.[*] Extract it and run ./configure. Please note that configure indicates missing libraries and packets that you may first need to install using the package system of your operating system.

$ tar -xzf wendzelnntpd-2.0.0.tgz
$ cd wendzelnntpd
$ ./configure

Please Note: If you wish to compile WITHOUT MySQL or WITHOUT SQlite support, then run MYSQL=NO ./configure or SQLITE=NO ./configure, respectively.

After configure finished, run make:

$ make

To install WendzelNNTPd on your system, you need superuser access. Run make install to install it to the default location /usr/local/*.

$ sudo make install

Please Note (Upgrades): Run sudo make upgrade instead of sudo make install for an upgrade. Please cf. Section 5.

Please Note (MySQL): If you plan to run MySQL, then no database was set up during 'make install'. Please refer to chapter 3 (configuration) to learn how to generate the MySQL database.

1 Init Script for automatic Startup

There is a init script in the directory scripts/startup. It uses the usual parameters like ``start'', ``stop'' and ``restart''.

2 Inofficial Note: Mac OS X

A user reported WendzelNNTPd-2.0.0 is installable under Mac OS X 10.5.8. The only necessary change was to add the flag ``-arch x86_64'' to compile the code on a 64 bit system. However, I never tried to compile WendzelNNTPd on a Mac.

3 Windows

Not supported.

3 Configuration

This chapter will explain how to configure WendzelNNTPd once it has been installed on your system.

Note: The configuration file of WendzelNNTPd is named /usr/local/etc/wendzelnntpd.conf. The format of the configuration file should be self describing and the standard config file includes many comments which will help you to understand the lines you can see and modify there.

Note: On *nix-like operating systems the default installation path is /usr/local/* what means that the configuration file of WendzelNNTPd will be /usr/local/etc/wendzelnntpd.conf, and the binaries will be placed in /usr/local/sbin.

1 Choosing an database engine

The first, and most important step, is to choose a database engine. You can either use SQlite3 (what is the default case and easy to use, but not very performant) or MySQL (what is of course the better solution, but also a little bit more complicated to realize). By default, WendzelNNTPd is configured for SQlite3 and is ready to run. If you want to keep this setting, you do not have to read this section.

1 Modifying the wendzelnntpd.conf

In your configuration file, you will find a line called database-engine. Here you can either write sqlite or mysql.

database-engine mysql

If you use MySQL, you also need to specify the user and password which WendzelNNTPd should use to connect to the server. If your server does not run on the localhost or uses not the default MySQL port, you have to modify these values too.

; Your database hostname (not needed for sqlite3)

; the database connection port (not needed for sqlite3)
; Comment out to use the default port of your database engine
database-port 3306

; Server authentication (not needed for sqlite3)
database-username mysqluser
database-password supercoolpass

2 Generating your database tables

Once you have chosen your database, you need to create the (database and) table in your database.

1 SQlite

In the case of SQlite, make install already did this for you.

Note: The SQlite database file as well as the posting management files will be stored in /var/spool/news/wendzelnntpd/.


For MySQL, a SQL script file is included called mysql_db_struct.sql. It creates the WendzelNNTPd database as well as all needed tables. Use the MySQL console tool to execute the script.

$ cd /path/to/your/extracted/wendzelnntpd-archive/
$ mysql -u YOUR-USER -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 48
Server version: 5.1.37-1ubuntu5.1 (Ubuntu)

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> source mysql\_db\_struct.sql
mysql> quit

2 Network Settings

Now you should specify the IP addresses (IPv4 or IPv6) for WendzelNNTPd to accept connections on, as well as the TCP port (the NNTP default port is 119) to run on.

; You have to specify the port _before_ using the 'listen' command!
port	119
; network addresses to listen on
listen	::1

You could also use different ports for different IP addresses by placing a port command right before each listen command but this is not recommended.

3 Setting the Allowed Size of Postings

To change the maximum size of a posting to be sent to the server, change the variable max-size-of-postings. The value must be set in Bytes and the default value is 20971520 (20 MBytes).

max-size-of-postings 20971520

4 Verbose Mode

If you have any problems running WendzelNNTPd or if you simply want more information about what is happening, you can uncomment the verbose-mode line.

; Uncomment 'verbose-mode' if you want to find errors or if you
; have problems with the logging subsystem. All log strings are
; written to stderr too, if verbose-mode is set. Additionaly all
; commands sent by clients are written to stderr too (but not to
; logfile)

5 Security Settings

1 Authentication and Access Control Lists (ACL)

WendzelNNTPd contains an extensive access control subsystem. If you want to allow only authenticated users the access to the server, you should uncomment use-authentication. This gives every authenticated user access to each newsgroup.

; Activate authentication

If you need a really advanced authentication system, you can activate Access Control Lists (ACL) by uncommenting use-acl. This activates the support for Role based ACL too.

; If you activated authentication, you can also activate access
; control lists (ACL)

2 Anonymized Message-ID

By default, WendzelNNTPd makes a users hostname or IP part of new message-IDs when a user sents a posting using the NNTP POST command. If you do not want that, you can force WendzelNNTPd not to do that by uncommenting enable-anonym-mids, what enables anonymized Message-IDs.

; This prevents that IPs or Hostnames will become part of the
; message ID generated by WendzelNNTPd what is the default case.
; Uncomment it to enable this feature.

4 Starting and Running WendzelNNTPd

1 Starting the Service

Once your WendzelNNTPd installation has been configured, you can run the server (in the default case you need superuser access to do that since this is required to bind WendzelNNTPd to the default NNTP port 119) by starting /usr/local/sbin/wendzelnntpd.

$ /usr/local/sbin/wendzelnntpd 
WendzelNNTPd-OSE (Open Source Edition): peak: 3, size_sockinfo: 1
WendzelNNTPd-OSE (Open Source Edition): version 1.9.0-pre-alpha 'Stockholm'  - (Mar  5 2010 21:36:19 #2326) is ready.

Note (Daemon Mode): If you want to run WendzelNNTPd as a background daemon process on *nix-like operating systems, you should use the parameter -d.

Note (Admininstration Tool): Use the command line tool wendzelnntpadm to configure user, role and newsgroup settings of your WendzelNNTPd installation. To get an overview of supported commands, run ``wendzelnntpadm help'':

$ wendzelnntpadm help
usage: wendzelnntpd <command> [parameters]
*** Newsgroup Administration:
 <addgroup | modgroup> <newsgroup> <posting-allowed-flag (y/n)>
 <delgroup> <newsgroup>
*** User Administration:
 <adduser> <username> [<password>]
 <deluser> <username>
*** ACL (Access Control List) Administration:
 <addacluser | delacluser> <username> <newsgroup>
 <addaclrole | delaclrole> <role>
 <rolegroupconnect | rolegroupunconnect> <role> <newsgroup>
 <roleuserconnect | roleuserunconnect> <role> <username>

2 Creating/Listing/Deleting Newsgroups

You can either list, create or delete newsgroups using wendzelnntpadm.

1 Listing existing newsgroups

$ wendzelnntpadm listgroups
Newsgroup, Low-, High-Value, Posting-Flag
alt.test 10 1 y
mgmt.talk 1 1 y
secret.project-x 20 1 y

2 Creating a new newsgroup

To create a new newsgroup run the following command:

$ wendzelnntpadm addgroup my.cool.group y
Newsgroup my.cool.group does not exist. Creating new group.

You can also change the ``posting allowed'' flag of a newsgroup but this takes currently no effect since WendzelNNTPd handles all newsgroups as newsgroups with posting allowed.

$ wendzelnntpadm modgroup my.cool.group y
Newsgroup my.cool.group exists: okay.
$ wendzelnntpadm modgroup my.cool.group n
Newsgroup my.cool.group exists: okay.

3 Deleting a newsgroup

$ wendzelnntpadm delgroup my.cool.group
Newsgroup my.cool.group exists: okay.
Clearing association class ... done
Clearing ACL associations of newsgroup my.cool.group... done
Clearing ACL role associations of newsgroup my.cool.group... done
Deleting newsgroup my.cool.group itself ... done
Cleanup: Deleting postings that do not belong to an existing newsgroup ... done

3 Administrating User Accounts

The easiest way to give only some people access to your server is to create user accounts (please make sure you activated authentication in your configuration file). You can add, delete and list all users.

1 Listing Users (and Passwords)

This command always prints you the password of a user too.

$ wendzelnntpadm listusers
Username, Password
developer1, wegerhgrhtrthjtzj
developer2, erghnrehhnht
manager1, wegergergrhth
manager2, thnthnrothnht
swendzel, lalalegergreg
swendzel2, 94j5z5jh5th
swendzel3, lalalalala
swendzel4, wegwegwegwegweg

2 Creating a new user

You can either enter the password as additional parameter (usefull for scripts that create users automatically) ...

$ wendzelnntpadm adduser UserName HisPassWord
User UserName does currently not exist: okay.

... or you can type it using the prompt (in this case the input is shadowed):

$ wendzelnntpadm adduser UserName2
Enter new password for this user (max. 100 chars):
User UserName2 does currently not exist: okay.

Please Note: A password must include at least 8 characters and may not include more than 100 characters.

3 Deleting an existing user

$ wendzelnntpadm deluser UserName2
User UserName2 exists: okay.
Clearing ACL associations of user UserName2... done
Clearing ACL role associations of user UserName2... done
Deleting user UserName2 from database ... done

4 Access Control List Administration (just in case, when the standard NNTP authentication is not enough)

Welcome to the advanced part of WendzelNNTPd. WendzelNNTPd includes a powerful role based access control system. You can either only use normal access control lists where you can configure which user will have access to which newsgroup. Or you can use the advanced role system: You can add users to roles (e.g. the user ``boss99'' to the role ``management'') and give a role access to a group (e.g. role ``management'' shall have access to ``discuss.management'').

Note: Please note that you must activate the ACL feature in your configuration file to use it.

Note: To see *ALL* data related to the ACL subsystem of your WendzelNNTPd installation, simply use ``wendzelnntpadm listacl''.

1 Invisible Newsgroups

WendzelNNTPd includes a feature called ``Invisible Newsgroups'' which means that a user without access to a newsgroup will neither see the newsgroup in the list of newsgroups, nor will he be able to post to such a newsgroup or will be able to read it.

2 Simple Access Control

We should start with the simple access control where you can define which user should have access to which newsgroup.

1 Giving a user access to a newsgroup

$ wendzelnntpadm addacluser swendzel alt.test
User swendzel exists: okay.
Newsgroup alt.test exists: okay.
$ wendzelnntpadm listacl
List of roles in database:

Connections between users and roles:
Role, User

Username, Has access to group
swendzel, alt.test

Role, Has access to group

2 Removing a user's access to a newsgroup

$ wendzelnntpadm delacluser swendzel alt.test
User swendzel exists: okay.
Newsgroup alt.test exists: okay.

3 Adding and Removing ACL Roles

If you have many users, some of them should have access to the same newsgroup (e.g. the developers of a new system should have access to the development newsgroup of the system). With roles you do not have to give every user explicit access to such a group. Instead you add the users to a role and give the role access to the group. (One advantage is that you can easily give the complete role access to another group with only one command instead of adding each of its users to the list of people who have access to the new group).

In the following examples, we give the users ``developer1'', ``developer2'', and ``developer3'' access to the development role of ``project-x'' and connect their role to the newsgroups ``project-x.discussion'' and ``project-x.support''. To do so, we create the three users and the two newsgroups first:

$ wendzelnntpadm adduser developer1
Enter new password for this user (max. 100 chars):
User developer1 does currently not exist: okay.
$ wendzelnntpadm adduser developer2
Enter new password for this user (max. 100 chars):
User developer2 does currently not exist: okay.
$ wendzelnntpadm adduser developer3
Enter new password for this user (max. 100 chars):
User developer3 does currently not exist: okay.

$ wendzelnntpadm addgroup project-x.discussion y
Newsgroup project-x.discussion does not exist. Creating new group.
$ wendzelnntpadm addgroup project-x.support y
Newsgroup project-x.support does not exist. Creating new group.

1 Creating a Role

Before you can add users to a role and before you can connect a role to a newsgroup, you have to create a role (you have to choose an ASCII name for it). In this example, the group is called ``project-x''.

$ wendzelnntpadm addaclrole project-x
Role project-x does not exists: okay.

2 Deleting a Role

You can delete a role by using ``delaclrole'' instead of ``addaclrole'' like in the example above.

4 Connecting and Disconnecting Users with/from Roles

To add (connect) or remove (disconnect) a user to/from a role, you need to use the admin tool too.

1 Connecting a User with a Role

The second parameter (``project-x'') is the role name and the third parameter (``developer1'') is the username. Here we add our 3 developer users from the example above to the group project-x:

$ wendzelnntpadm roleuserconnect project-x developer1
Role project-x exists: okay.
User developer1 exists: okay.
Connecting role project-x with user developer1 ... done
$ wendzelnntpadm roleuserconnect project-x developer2
Role project-x exists: okay.
User developer2 exists: okay.
Connecting role project-x with user developer2 ... done
$ wendzelnntpadm roleuserconnect project-x developer3
Role project-x exists: okay.
User developer3 exists: okay.
Connecting role project-x with user developer3 ... done

2 Disconnecting a User from a Role

$ wendzelnntpadm roleuserconnect project-x developer1
Role project-x exists: okay.
User developer1 exists: okay.
Connecting role project-x with user developer1 ... done

5 Connecting and Disconnecting Roles with/from Newsgroups

Even if a role is connected with a set of users, it is still useless until you connect the role with a newsgroup.

1 Connecting a Role with a Newsgroup

To connect a role with a newsgroup, we have to use the command line tool for a last time (the 2nd parameter is the role, and the 3rd parameter is the name of the newsgroup). Here we connect our ``project-x'' role to the two newsgroups ``project-x.discussion'' and ``project-x.support'':

$ wendzelnntpadm rolegroupconnect project-x project-x.discussion
Role project-x exists: okay.
Newsgroup project-x.discussion exists: okay.
Connecting role project-x with newsgroup project-x.discussion ... done
$ wendzelnntpadm rolegroupconnect project-x project-x.support
Role project-x exists: okay.
Newsgroup project-x.support exists: okay.
Connecting role project-x with newsgroup project-x.support ... done

2 Disconnecting a Role from a Newsgroup

This is done like in the example above but you have to use the command ``rolegroupunconnect'' instead of ``rolegroupconnect''.

6 Listing your whole ACL configuration again

Like mentioned before, you can use the command ``listacl'' to list your whole ACL configuration (except the list of users that will be listed by the command ``listusers'').

$ wendzelnntpadm listacl
List of roles in database:

Connections between users and roles:
Role, User
project-x, developer1
project-x, developer2
project-x, developer3

Username, Has access to group
swendzel, alt.test

Role, Has access to group
project-x, project-x.discussion
project-x, project-x.support

1 Saving time

As mentioned above, you as the maintainer of the usenet server can safe time by using roles. If you add a new developer to the system, and the developer should have access to the two groups ``project-x.discussion'' and ``project-x.support'', you do not have to assign the user to both (or even more) groups. Instead, you just add the user to the role ``project-x'' that is already connected to both groups.

If you want to give all developers access to the group ``project-x.news'', you also do not have to connect every developer with the project. Instead, you just connect the rule with the newsgroup, what is one command instead of $n$ commands. Of course, this time saving concept also works if you want to delete a user.

5 Hardening

Besides the already mentioned authentification, ACL and RBAC features, the security of the server can be improved by putting WendzelNNTPd in a chroot environment or letting it run under an unpriviledged user account (the user then needs write access to /var/spool/news/wendzelnntpd and read access to (/usr/local)/etc/wendzelnntpd.conf). An unpriviledged user under Unix-like systems is also not able to create a listen socket on the default NNTP port (119) since all ports up to 1023 are reserved. This means that the server should use a port >= 1024 if it is started by a non-root user. Note: Some Unix systems may have another priviledged port configuration.

5 Upgrade to a newer version

1 Upgrade from 2.0.x to 2.0.y

Stop WendzelNNTPd if it is currently running. Install WendzelNNTPd as described but run make upgrade instead of make install. Afterwards, start WendzelNNTPd again.



2 Upgrade from v.1.4.x to 2.0.x

Acknowledgement: I would like to thank Ann from Href.com for helping a lot with finding out how to upgrade from 1.4.x to 2.0.x!


An upgrade from version 1.4.x was not foreseen due to the limited available time I have for the development. However, here is a dirty hack:

First Step:

You need to install Wendzelnntpd-2.x on a Linux system. This requires some libraries and tools. Under Ubuntu they all come as packages: sudo apt-get install libmysqlclient-dev, libsqlite3-dev, flex, bison, sqlite3. Under CentOS they come as packages as well: sudo yum install make gcc bison flex sqlite-devel. Other operating systems should provide the same or similar packages/ports.

Run MYSQL=NO ./configure, make, and sudo make install. This will compile, build and install WendzelNNTPd without MySQL support as you only rely on SQLite3 from v.1.4.x.

Second Step:

Please make sure WendzelNNTPd-2 is configured in a way that we can *hopefully* make it work with your own database file. In /usr/local/etc/wendzelnntpd.conf make sure that WendzelNNTPd uses sqlite3 instead of mysql:

database-engine sqlite3

Third Step:

Now comes the tricky part. The install command should have created /var/spool/news/wendzelnntpd/usenet.db However, it is an empty usenet database file in the new format. Now REPLACE that file with the file you use on your existing WendzelNNTPd installation, which uses the old 1.4.x format. Also copy all cdp* files and the nextmsgid file from your Windows directory to /var/spool/news/wendzelnntpd/.

The following step is a very dirty hack but I hope it works for you. It is not 100% perfect as important table columns are then still of the type 'STRING' instead of the type 'TEXT'!

Load the sqlite3 tool with your replaced database file:

$ sudo sqlite3 /var/spool/news/wendzelnntpd/usenet.db

This will open the new file in editing mode. We now add the tables which are not part of v.1.4.x to your existing database file. Therefore enter the following commands:

CREATE TABLE users2roles (username TEXT, role TEXT, PRIMARY KEY(username, role));
CREATE TABLE acl_users (username TEXT, ng TEXT, PRIMARY KEY(username, ng));
CREATE TABLE acl_roles (role TEXT, ng TEXT, PRIMARY KEY(role, ng));

Fix Postings

You will probably see no posting bodies right now if postings are requested by your client. Therefore, switch into /var/spool/news/wendzelnntpd and run (as superuser) this command, it will replace the broken trailings with corrected ones:

for filn in `/bin/ls cdp*`; do echo $filn; cat $filn | \
sed 's/\.\r/.\r\n/' > new;  num=`wc -l new| \
awk '{$minone=$1-1; print $minone}'` ; \
head -n $num new > $filn; done

Last Step (Checking whether it works!):

First check, whether the datbase file is accepted at all:

$ sudo wendzelnntpadm listgroups

It should list all your newsgroups

$ sudo wendzelnntpadm listusers

It should list all existing users. Accordingly

$ sudo wendzelnntpadm listacl

should list all access control entries (which will be empty right now but if no error message appears, the related tables are now part of your database file!).


Now start WendzelNNTPd via sudo wendzelnntpd and try to connect with an NNTP client to your WendzelNNTPd and then try reading postings and try sending new postings and try replying to these.

If this works, you can now run v2.x on 32bit and 64bit Linux :)


Starting WendzelNNTPd
4 | 1

About this document ...

WendzelNNTPd-OSE Version 2 Documentation

This document was generated using the LaTeX2HTML translator Version 2008 (1.71)

Copyright © 1993, 1994, 1995, 1996, Nikos Drakos, Computer Based Learning Unit, University of Leeds.
Copyright © 1997, 1998, 1999, Ross Moore, Mathematics Department, Macquarie University, Sydney.

The command line arguments were:
latex2html -show_section_numbers -local_icons -split 2 docs.tex

The translation was initiated by Steffen Wendzel on 2015-09-26

Steffen Wendzel 2015-09-26