Firma CHANGELOG --------------- 29/07/2007 - 0.3-cvs (rev 1.168) - luis - firma would always exit successfully (exit code = 0). Fixed. - Message signature checking was being completely skipped. Fixed. - Passphrase check for sequentially repeated characters wasn't working. Fixed. - Trying to create a list with the same name (same homedir path, actually) of an existing list would cause this list to be removed. Fixed. - firma's user and group would always be set to nobody:nobody, disregarding what was entered at firma.conf. Fixed. - Not all global variables were being unset at the end of firma's execution. Fixed. - Better list creation routine. Minor bugs fixed. 31/01/2007 - 0.3-cvs (rev 1.152) - luis - Removed firma dependecy on bc, cat and tac. 30/01/2007 - 0.3-cvs (rev 1.151) - luis - Re-enabling passphrase complexity checking. Not sure why I had commented out this part of the code. - s/Copyright (C) 2005/Copyright (C) 2005-2007/ - 1,$s/ERROR_MESSAGE=/LogMessage /. This way messages are automatically echoed/logged, allowing firma to deal with more than one error message on the same execution. So ERROR_MESSAGE is no more. - Removed firma dependecy on awk and test/[. 19/01/2007 - 0.3-cvs (rev 1.150) - rhatto Added command "listinfo" 12/11/2006 - 0.3-cvs (rev 1.148) - rhatto Increased SubscribeUsers verbosity; tagged as 0.3pre1 :) 18/10/2006 - 0.3-cvs (rev 1.145) - luis Fixed major bug: PGP/MIME handling would work only if variable REMOVE_THESE_HEADERS_ON_ALL_LISTS was not empty. 13/10/2006 - 0.3-cvs (rev 1.143) - rhatto Lots of stuff for today :) - new list config parameters: - DELIVERY_RANDOMIZATION: if non-zero, set a random delay between 0 and N seconds between each messsage delivery; if you run firma with a TLS-enabled MTA and mostly of the list messages are sent to others TLS-enabled MTAs, then this option will make harder to a sniffer detect the traffic of you mailing list, specially if your MTA already sends a lot of messages or if you're going to have a lot of encrypted mailing lists, all randomizing its delivery. - REPLAY_PROTECTION: when set to "yes", stores sha1sums of the last REPLAY_COUNT received messages; then, if some message with an already stored sha1sum, then its bounced back to the sender and considered as an attempt of replay attack. - REPLAY_COUNT: number of messages to store sha1sums; defaults to 10 and only used when REPLAY_PROTECTION is set to "yes". - REPLAY_FILE: file to store sha1sums of messages; only used when REPLAY_PROTECTION is set to "yes"; defaults to /var/log/firma/replay.db. - in function NewList: - changed variable DESCRIPTION to KEY_DESCRIPTION. - fixed key lenght size. - now key information (size and description) is written in the list config file; this can help in the future as key regeneration support is added. - fix in config parameters evaluation - new list config variable LIST_REQUEST_ADDRESS that defaults to listname-request@domain. - in function MimeWrapMessage: changed From: and Reply-To: to $LIST_REQUEST_ADDRESS. - new functions: - SetDeliveryRandomization - DeliveryRandomization - ReplayProtectionFlush - ReplayProtectionCheck - in function ReEncryptAndSendListMessage: added a call to DeliveryRandomization both when hidden recipient is enabled or disabled. - in function ProcessMessage: added call to ReplayProtectionCheck and else clause. 12/10/2006 - 0.3-cvs (rev 1.138) - rhatto - new config file evaluation scheme: now config files aren't sourced to firma script but instead every parameter is evaluated; this makes config evaluation more secure. - all help on admin commands can be obtained by command line: firma --help subscribe firma --help sendkey - removed config parameters information at the header of the script; now this message can be accessed by command line: firma --help config - new functions: - EvalConfigParam - SourceFirmaConfig - SourceListConfig - ConfigHelp - AdminHelp 12/10/2006 - 0.3-cvs (rev 1.127) - rhatto - now admin messages are being encrypted and then sent back to the administrator. - restored function MimeWrapMessage. - new function CreateMessageBodyPart. 11/10/2006 - 0.3-cvs (rev. 1.114) - rhatto - support of list administration via email - in function ParseGpgDecryptStderr: dded variable SIGNATURE_MADE_BY_SENDER, that checks wheter the signature was made by the sender of the email; this is important to check if admin tasks are really being request by the list administrator, etc. - new list config option REQUIRE_SIGNATURE= wheter messages sent to the list should be (yes) or dont need to be signed to be processed (no); defaults to yes; this doesnt affect the way email administration works, when signature is mandatory. - in function ProcessMessage: - fixed typ0 LIST_ADMINS (should be LIST_ADMIN). - check whether SIGNATURE_MADE_BY_SENDER is true. - process message according value of MODE, so the function can process either normal list messages or admin commands. - in function ChooseUid: added expect exit status checking at the bottom of the function. - new command line option -e, --email-admin-task. - new global variable MODE is set either - "admin-interactive". - "admin-non-interactive". - "list-message". - new functions - AdminLog - EmailListAdministration - AllowMessageProcessing 10/10/2006 - 0.3-cvs (rev 1.89) - rhatto - minor fixes - new variables GPG_FLAGS_NO_BATCH and GPG_NO_BATCH for --no-batch processing. - in function NewList: - atomic list creation: if something fails, $LIST_HOMEDIR is erased. - fixed HOMEDIR evaluation. - input sanitizer. - can automatically create a passphrase. - can import admins pubkeys. - can send list pubkey to admins. - key size. - key expiration. 09/10/2006 - 0.3-cvs (rev 1.74) - rhatto - new admin commands: - "list" show list subscribers. - "subscribe" to add users to the list. - "sendkey" to send the list pubkey to a given user. - "sendkey" to send the list pubkey to a given user. - "info" get a subscriber pubkey information. - changed "quit" exit code on ListAdministration to 3 so the main function doesnt get confused between a gpg error (that usually is status code 1 or 2) and the list administration quit command. - new firma.conf and list.conf variable KEYSERVER= default keyserver to import/export keys (defaults to keyserver.noreply.org). - fixes on permission checking. - some other small fixes. - new functions: - UnsubscribeUser - SubscribeUsers - SendListPubkey - GetSubscribersInfo - FixListOwnership - RandomString - function NewList now asks whether to send or not the list pubkey to the admins. 07/10/2006 - 0.3-cvs (rev 1.49) - rhatto - new function LogMessage. - new function prototype UnsubscribeUser. - fixes on permission checking. 06/10/2006 - 0.3-cvs (rev 1.43) - rhatto - user and group verification added just in the bottom of function CheckFirmaConfigFile through firma or list configuration parameters USER and GROUP. - new global variable BASENAME contains firma program name. - NewList: now set permission and ownership to $USER and $GROUP in all files of the list folder. - new function CheckPermission. - new function CheckListPermissions. - permission checking when evaluation firma.conf and list config file. - README update 29/09/2006 - 0.3-cvs (revs. 1.37 - 1.4.2) - luis - Log to syslog: firma.conf related variables: LOG_TO_SYSLOG (on if set to "1") LOGGER_BINARY (path to logger's binary) SYSLOG_PRIORITY (defaults to "user.err") - Minimal message munging: By not breaking the MIME structure of messages, firma is now as PGP/MIME (and PGP/Inline) compliant as the MUAs used by the list subscribers. - Headers: Message headers are not edited in any way unless firma/the list is set to do so. firma.conf related variable: REMOVE_THESE_HEADERS_ON_ALL_LISTS (space separated case-insensitive entries) (may include regexps (e.g., X-.*) list.conf related variable: REMOVE_THESE_HEADERS (same as above) - Two new list configuration parameters SUBJECT_PREFIX (prefix to be included in the subject of list messages) REPLIES_SHOULD_GO_TO_LIST (if set to "1", a Reply-To header containing the list address is added to all list messages) - Body: firma only replaces the original encrypted block with a new one, encrypted to all list members. The rest of the message body, if any, is left untouched. - Bounce processing: A list can now be set to silently ignore invalid messages, instead of sending bounces as it normally does. list.conf related variable: SILENTLY_DISCARD_INVALID_MESSAGES (on if set to "1") - Non-fatal errors: Not all errors thrown by firma are fatal, as it used to be. Less critical errors just print/log a warning now. - Message sending: Messages are sent to all recipients at once. The addresses are passed as arguments to the mail program, so firma no longer depends on the To, Cc, Bcc headers of the message being sent. These headers are also left untouched on the original message. - Better performance: firma now processes both valid and invalid messages faster. 04/09/2005 - 0.3-cvs (rev 1.36) - luis - As a temporary solution until compliance to PGP/MIME standards can be implemented, messages are being decoded using the command "mimencode -q -u". This command assumes that the entire message is Quoted-Printable encoded. So, for now, no Base64 encoded messages should go through firma. - Message's signature checking is being more thoroughly parsed so that firma can send more specific bounce messages. Added three different messages, informing the sender either that his/her address is not subscribed to the list, or that the received message was not signed, or that it was not encrypted with the list's public key. - Signature checking output, as displayed in the top of the list messages, is more verbose now, showing all UIDs of the signing key instead of just the primary one. - Expired, revoked or disabled public keys are no longer processed when checking if a given list has any subscribers or when a message is being sent to the list. - GetSubscribersList output is now stored in variable SUBSCRIBERS_LIST for better performance of the functions depending on it. For the same reason, GetGpgDecryptStderr output is now stored in variable GPG_DECRYPT_STDERR. - Major changes on functions SendListMessage, SendWarningMessage, SendBounceMessage and ProcessMessage to implement all the items above. - Re-declared global array MESSAGE_BODY as a global variable and added/removed some other variables: recipients -> ( removed ) ( new ) -> GPG_DECRYPT_STDERR, SUBSCRIBERS_LIST - Added two small routines to declare/unset all global variables. 03/09/2005 - 0.3-cvs (rev 1.35) - luis - In the PASSPHRASE, characters can now be sequentially repeated up to 4 times instead of 3. 02/09/2005 - 0.3-cvs (rev 1.34) - luis - Added two optional (and experimental) parameters to firma.conf: USE_GPG_HIDDEN_RECIPIENT_OPTION and SEND_MESSAGES_USING_BCC. If both are enabled, firma will use GnuPG's --hidden-recipient option (available from version 1.4.0 onwards) and send list messages to all subscribers at once, using BCC. This should speed up message processing, specially on lists with several subscribers. - Major changes on functions SendListMessage and SendWarningMessage to implement the two options above, and some minor changes on functions SendBounceMessage and ProcessMessage. - Added a new check on function CheckFirmaConfigFile, to make sure GnuPG's version is 1.4.0 or later in case USE_GPG_HIDDEN_RECIPIENT_OPTION is enabled. - Added/removed some local variables: email -> ( removed ) ( new ) -> subscriber, email_address, recipients - Corrected misspelled words and fixed typos on comments. 30/08/2005 - v0.2.5 (rev 1.33) - rhatto - Added GPL copyright notice and firma contact email 23/08/2005 - 0.3-cvs (rev 1.32) - luis - Fixed a possible infinite loop (added on rev 1.30) on function GetMessageHeaders. It could be triggered, for example, by a message without a body. - Like the other header variables (FROM, SUBJECT), the variable DATE now holds only the given date and no longer the entire "Date:" line. 17/08/2005 - 0.3-cvs (rev 1.31) - luis - On function GetMessage, renamed variable "STDIN" to "stdin" and declared it as local. 17/08/2005 - 0.3-cvs (rev 1.30) - luis - Changed project description to: "firma: GnuPG-based encrypted mailing list manager" - Added/renamed/removed variables and arrays: o Configuration file variables: MAIL -> MAIL_AGENT MAIL_ARGS -> MAIL_AGENT_ARGS GPG -> GPG_BINARY LISTNAME -> LIST_ADDRESS LISTADMIN -> LIST_ADMIN GPGDIR -> LIST_HOMEDIR PASSWD -> PASSPHRASE o GnuPG variables: GPGFLAGS -> GPG_FLAGS GPGCOMMAND -> GPG GPGLIST -> GPG_LIST_KEYS GPGDECRYPT -> GPG_DECRYPT GPGENCRYPT -> GPG_ENCRYPT o Other global variables: FIRMA_LIST_PATH -> LISTS_DIR FROMADD -> SENDER_ADDRESS array -> ARRAY CONFIG_FILE -> LIST_NAME CONFIG_PATH -> LIST_PATH CONFIG -> LIST_CONFIG_FILE LINES, n, i -> ( removed ) ( new ) -> FIRMA_CONFIG_FILE o Global arrays: MESSAGE -> ORIG_MESSAGE GPG_MESSAGE -> ORIG_GPG_MESSAGE LIST_MESSAGE -> MESSAGE_BODY USED_ARRAYS -> GLOBAL_ARRAYS ADMINCOMMANDS -> ( removed ) ( new ) -> ORIG_MESSAGE_HEADERS o Local variables: ADMIN -> administrator EMAIL -> email KEYID -> keyid signal -> ( removed ) ( new ) -> element, i, j, uid_count, chosen_uid_number - firma now uses two different configuration files: a general one, containing the variables MAIL_AGENT, MAIL_AGENT_ARGS, GPG_BINARY and LISTS_DIR, and a list specific file, containing the variables LIST_ADDRESS, LIST_ADMIN, LIST_HOMEDIR and PASSPHRASE. - Edited the comments at the beginning of the script to reflect the change above. - As suggested by the Advanced Bash-Scripting Guide (ABSG)[1], added descriptive headers to all functions, describing its function, what it expects as input, on what other functions it depends and what exit codes it returns. - Also as suggested by the ABSG[1], renamed all functions from all_lower_case_names to MixedCaseNames. - Renamed some functions to make their use more clear: gpg_args -> DeclareGpgVars check_config -> split into two: CheckFirmaConfigFile and CheckListConfigFile get_gpg_stderr -> GetGpgDecryptStderr message_list -> SendListMessage message_list_error -> SendWarningMessage message_list_return -> SendBounceMessage list_admin -> ListAdministration - Added some more GnuPG flags: o To the GPG_FLAGS variable: --no-options, --no-default-keyring, --no-auto-check-trustdb o And to the GPG_ENCRYPT variable: --local-user $LIST_ADDRESS, --no-emit-version, --trust-model always The last one replaced the "--always-trust" option, since its deprecated according to GnuPG's manual. - Minor changes on functions: Usage, Check*ConfigFile, GetMessage, GetSubscribersList, SendListMessage, SendWarningMessage, SendBounceMessage, ProcessMessage, NewList and ChooseUid o Usage: Explained what options expect an argument; Commented out the -r (--list-request) option description, since it's not implemented yet; Added a description of the administrative tasks accepted by the -a option; Other minor changes. o Check*ConfigFile: Added a check to see if the LISTS_DIR is an existing directory; Added a check to avoid more than 3 sequential repetitions of the same character in the PASSPHRASE; Other minor changes in the PASSPHRASE, LIST_ADDRESS and LIST_ADMIN checks. o GetMessage: Added a check to see if the message was successfully stored in the ORIG_MESSAGE array. o GetSubscribersList: Added a check to see if there are any subscribers to send messages to in a given list. o SendListMessage, SendWarningMessage, SendBounceMessage: Just minor changes. o ProcessMessage: (* SECURITY FIX *) A public key with an UID containing GOODSIG in its name, comment or email address would be able to send messages to any list. And a UID containing BADSIG in any of these fields, would be able to send messages to the list administrator(s) of any list. Fixed. o NewList: Just minor changes. o ChooseUid: Declared keyid, uid_count and chosen_uid_number as local variables, moving them to the beginning of the function; Minor syntax changes in the checks; Added lots of comments, explaining what the checks and the expect script are doing. - Major changes on functions: GetGpgMessage, GetMessageHeaders, ListAdministration and "main" o GetGpgMessage: Improved function to run faster, specially when processing large messages (over 50KB or so). o GetMessageHeaders: Improved to run faster independent of the size of the message being processed. o ListAdministration: Structured function using the "case" bash builtin; Made command parsing more consistent: checking if a given command exists, if the right number of arguments were passed, if the arguments are valid for this command, and, if anything is wrong, showing descriptive and concise error messages; New administrative tasks can be easily added to this new nested "case" structure. o "main": Also structured using the "case" bash builtin; The long options --admin-task, --create-newlist, --help, --process-message and --version are now accepted; Commented out the -r (--list-request) option, since it's not implemented yet; Made the command-line options parsing more consistent (read explanation above); Changed "umask" from 0777 to 0077. Otherwise, function NewList cannot create configuration files for new lists; Improved parsing of option -a (--admin-task), removing unnecessary commands and routines from this part of the code; Again, new command-line options can be easily added to this new nested "case" structure. [1] http://www.tldp.org/LDP/abs/html/unofficialst.html 10/08/2005 - 0.3-cvs (rev 1.29) - rhatto - Fix: Now LIST_MESSAGE _really_ works 09/08/2005 - 0.3-cvs (rev 1.28) - rhatto - Deleted var 'x' 07/08/2005 - 0.3-cvs (rev 1.19) - rhatto - Main function organization - umask 0777 to prevent any files got wrong permissions - Created USED_ARRAYS containing all the used array names - Small routine to unset all arrays 07/05/2005 - 0.3-cvs (rev 1.13) - rhatto - firma now uses sendmail or any smtp wrapper directly: - Specify your smtp wrapper through MAIL var, e.g.: MAIL=/usr/sbin/sendmail MAIL=/usr/bin/ssmtp - Command-line arguments passed to the wrapper must be specified in var MAIL_ARGS - Change in functions get_message and get_gpg_message, introducing the variable LINES (total lines of the received message). - New array: LIST_MESSAGE: keeps the encrypted message sent to each subscriber; needed to the smtp wrapper and will help PGP/MIME handling. 05/08/2005 - 0.3-cvs (rev 1.6) - luis - New functions: list_admin, choose_uid and version. - Renamed functions GPGSTDERR -> get_gpg_stderr SUBSCRIBERS -> get_subscribers_list get_headers -> get_message_headers - New args: -h: calls function usage -v: calls function version - Option -a passes STDIN line by line to list_admin - Added some checks in the main procedure - Some minor syntax changes 02/08/2005 - 0.3-cvs - rhatto - Completed function newlist (still needs tests) - New function: gpg_args 25/05/2005 - v0.2.4 - rhatto - Message is now stored in two arrays, MESSAGE and GPG_MESSAGE. Temporary files are no longer needed. Deleted TMP var. - Adapted firma to work with these two arrays. Lots of changes along the code. - Reorganized the code again, splitting part of the process_message function into three new functions (get_message, get_gpg_message and get_headers). - new functions: get_message, get_gpg_message, get_headers, - added export LANG=en_US 14/05/2005 - v0.2.3 - rhatto - Changed project description. - Two new variables: VERSION and CONFIG. The later stores the path to the configuration file. - Reorganized the code, splitting it into two functions (check_config and process_message). - An exit code of 1 is now given if an error occurs. - Implemented a simple command line syntax: firma now has to be called along with an option (e.g., -p to process a message), followed by the path to the configuration file to be used. - Created a new function (usage), which displays firma's usage if its called without a command line option. - Rearranged some comments along the code. - New functions are: check_config, eval_config, usage, process_message and splited the code into these routines 12/05/2005 - v0.2.2 - luis major code cleanup and bug fix release - New: Implemented size and complexity checks for the lists' passphrase. - firma would not work as expected if the PASSWD variable contained characters such as $, `, ', ", \. Fixed. Now PASSWD has to be enclosed in single quotes and cannot contain any single quote in its value. - Checked the entire code and tried to make it clearer and more consistent. Lots of small changes and minor fixes. - BASH was looking for the configuration file first on its PATH and only then on the current directory. Disabled sourcepath to avoid this behavior. - A From header containing more than one "<" would turn the variable FROMADD useless. Fixed. - When a message's signature is found invalid, the notification about it is now sent separately for the sender and the list administrators. - Other minor fixes and improvements. - Fix LISTNAME and LISTADMIN case sensitiveness. 18/04/2005 - v0.2.1 - luis - Added some simple checks for the values entered in the configuration file. If any of these checks fail, firma exits and shows what should be corrected in the file. - GPGLIST now lists public keys using GnuPG's option "--with-colons". Definitely much easier to parse. - added --no-permission-warning on $GPGCOMMAND. - first tests with --hidden-recipient, but not added - Fixed the FROMADD variable. It wasn't working on the last version - FROMADD=$(echo $FROMADD \ | if grep -q "<" ; \ then echo $FROMADD | grep -o "<.*>" | sed -e "s/[<>]//g" ; \ else echo $FROMADD ; \ fi) + FROMADD=$(echo $FROM \ | if grep -q "<" ; \ then echo $FROM | grep -o "<.*>" | sed -e "s/[<>]//g" ; \ else echo $FROM ; \ fi) - created routines to check config file vars - changed SUBJECT to SUBJECT=$(grep -m 1 "^Subject:" $TMP | cut -d : -f 2- | sed "s/^ //") - changed FROM to FROM=$(grep -m 1 "^From:" $TMP | cut -f 2 -d :) 17/04/2005 - v0.2 - luis - Lists can now have a passphrase. Added option "--passphrase-fd 0" to the GPGDECRYPT and GPGENCRYPT variables. - Added a new variable to the configuration file: LISTADMIN. It should contain the email address of one or more list administrator(s). - Added signature checking for received messages. If the signature is found valid, the message is sent to the list. If its invalid, its returned to the sender and to the list administrator(s). If the message was not encrypted and/or signed, a note is returned to the sender. - Added a few options to the GPGCOMMAND variable: "--batch", "--no-tty", "--no-use-agent" and "--no-permission-warning". GPGCOMMAND="$GPG \ --quiet \ --homedir $GPGDIR \ --batch \ --no-tty \ --no-use-agent" - Added two new functions: GPGSTDERR and SUBSCRIBERS. The first gets GnuPG's STDERR and discards its STDOUT and the later returns the subscribers list. function GPGSTDERR { echo "$PASSWD" \ | ($GPGDECRYPT --status-fd 2 $TMP.gpg 1> /dev/null) 2>&1 ; } function SUBSCRIBERS { $GPGLIST \ | sed -n '/$LISTNAME/d;/pub/p' \ | grep -o '<.*>' \ | sed -e 's/[<>]//g' ; } - Added a new variable, FROMADD, which contains the sender's email address. FROMADD=$(echo $FROMADD \ | if grep -q '<' ; \ then echo $FROMADD | grep -o '<.*>' | sed -e 's/[<>]//g' ; \ else echo $FROMADD ; \ fi) TODO: find a better place to this var. Feb 2005 - v0.1 - rhatto - First public release and proof of concept. - Based on pgplist (http://www.rediris.es/app/pgplist/index.en.html), but completely written from scratch. - Strips all mail headers. - Use keyring as database. - List variables are sourced from a configuration file. - List subscription is managed through GnuPG's public keyrings, one keyring for each mailing list. - Message is read from STDIN, passed to it by postfix or some other MTA. - Encrypted message is saved in a temporary file on disk. The decrypted message though is not stored in disk, but passed to the MTA through a pipe. - Message is then decrypted, re-encrypted and sent separately to each list subscriber.