0004693: multiple occurences of use_gpg lead to multiple checks

Notes
(0008819) bonivart (developer) 2011-02-21 18:48	The problem is not that you have use_gpg twice, it's that you have the same mirror twice. Usually options in /etc override those in /opt to allow for local modifications but the mirror option is special to allow for multiple mirrors. You declared the same mirror twice and they get checked twice. This has not changed in a while, I just tested it with 1.9 and the same thing happens there. I guess pkgutil should be smart enough to not register the same mirror more than once. I'll introduce some logic for that. Thanks for reporting it. For now you can just choose where to declare the mirror instead of doing it in both files. The general idea is that global, perhaps read-only, configurations are in /opt/csw/etc/pkgutil.conf and your local changes are in /etc/opt/csw/pkgutil.conf. If you have full control over the machine you edit /etc/opt/csw/pkgutil.conf and leave /opt/csw/etc/pkgutil.conf as it is.

(0008822) bonivart (developer) 2011-02-22 13:53	I would really like you to test my suggestion of setting the mirror in only one of the files and confirm that it is in fact the issue. I will keep the bug open for a few more days.

(0008823) flod (reporter) 2011-02-22 14:52	Okay, I will play a bit tonight when I have time. I have no idea what additional repositories (that make sense) one could add. Maybe one can use experimental repositories but I never tried that via config. If there would be stable/unstable opencsw branches it would be quite useful.

(0008824) bonivart (developer) 2011-02-22 17:11	I'm not really sure what you mean now. What I am pretty sure about is that you have set csw.informatik.uni-erlangen.de/csw/current in both /opt/csw/etc/pkgutil.conf and /etc/opt/csw/pkgutil.conf. You should only have the same mirror set in one of the files. So just comment it out (with a hash (#) mark) in one of the files (the one you usually don't change).

(0008826) bonivart (developer) 2011-02-23 19:40	In revision 361 I have added a detection of duplicate mirror definitions, it will skip them and alert you about which definition is defined multiple times. "Duplicate mirror definition skipped (http://mirror.opencsw.org/opencsw/unstable)." [^]

(0008828) flod (reporter) 2011-02-25 13:58	I can confirm that you are right. After removing duplicate entries I only get one gpg block. It is exactly as you described it, for every mirror= line one gets 3 additional lines with use_gpg=true. I never tried to add experimental repositories, but these work there too (but there seems to be no key that one can install). Now back to the reason out of which i stumbled upon that. I use something like: ... pkgutil -c 2> "$ERRFILE" > "$OUTFILE" ... cat "$OUTFILE" \| tail +3 \| grep -v SAME > "$RESFILE" to get automated daily update reports. The fixed number in the tail command is the problem. One solution would be that I just use a more intelligent shell script with sed. But the question is still, what stable interface can one expect from pkgutil for shell scripts. It would help if one could rely on something like always one header line and then the content on stdout. And all gpg/check output goes to stderr. The state now is: stdin - n lines info output (1 per mirror=), 1 header line, content follows stderr - gpg output, errors And it hit me that n was not stable (whatever caused the duplicate entries).

(0008829) flod (reporter) 2011-02-25 14:14	Something I would like to add about your proposed change - I guess you filter just by url and not by catalog content or something else. That way it is still possible to have duplicates of this kind: * same mirror but slightly other url (symbolic link, non canonical url ..) * another mirror So I don't know how much this special handling will help. Will the message about the skipped host be on stdout or stderr :-) ?

(0008830) bonivart (developer) 2011-02-25 14:53	There is a problem with the gpg setting being global and OpenCSW doesn't sign the experimental catalog. I'm looking into adding support for this and maybe more to be per mirror. Regarding your script output I think what you want has been in pkgutil for long now. First of all, you can use -C instead of -c to get rid of those SAME line. In addition to that there's the -e option to e-mail just this kind of report, I run it daily from root's crontab: 10 5 * * * /opt/csw/bin/pkgutil -U -e mail@foo.bar > /dev/null 2>&1 I just tested it with gpg on and it still just mails a list of upgradeable packages like this (spacing probably weird here in Mantis): CSWbash 4.1.7,REV=2010.11.20 4.1.9,REV=2010.12.20 CSWbind 9.7.2P3,REV=2010.12.03 9.7.3,REV=2011.02.23 CSWbindutils 9.7.2P3,REV=2010.12.03 9.7.3,REV=2011.02.23 CSWcas-etcservices 1.42,REV=2010.11.26 1.42,REV=2011.02.16 CSWgdbm 1.8.3,REV=2011.01.27 1.8.3,REV=2011.02.01 Regarding stable interface for scripting I have gotten feedback from more and more people that are using pkgutil for different things, it's hard for me to anticipate everything people do. :-) I have started (in 2.3) to introduce a --parse option as addition to -a/c/C and so on which removes the header and separates all columns with one tab. Give that a try. The duplication detection is for the exact same mirror path, I can't know the contents of the mirror. Assume you're creating your own local mirror (with bldcat) and you call that unstable to match our main mirrors. The message is printed to stderr, I'm trying to do this more strict now.

(0008831) flod (reporter) 2011-02-25 17:35	I did not look into the pkgutil options for a long time now. Thanks for your suggestions. I can not use -e as reporting is done with another framework on various platforms. I depend on stdout/stderr. pkgutil -C --parse 2> /dev/null is already better. But still the number of "Checking integrity of ...." lines there is variable.

(0008832) bonivart (developer) 2011-02-25 17:48	How about this? pkgutil -C --parse 2> /dev/null \| grep -v "^Checking integrity" I would prefer not move the "Checking" line to stderr since it's normal info, not a problem. On the other hand gpg uses stderr...

(0008836) flod (reporter) 2011-02-25 18:11	I have no problem using / creating complex regexes here. And I guess my local problem is already solved. I just wondered why these "Checking integrity of ...." do appear with --parse. It just felt not exactly what one would expect from parsable stdout. I did not know that this option exists before you mentioned it. Without --parse it really is normal information. So maybe you want to consider my remark on that, but the "bug" can be closed anyway.

(0008837) bonivart (developer) 2011-02-25 18:27	Ok, I think you're right. Parse is a new option so it may take some feedback to get it right. I will silence that "Checking..." line if --parse is on but gpg will still go to stderr, some may grep for that and not like it being gone, I'm not sure.

(0008838) flod (reporter) 2011-02-25 18:41	I already filter stderr out and only look at it when $? is not 0. I have no feelings towards gpg being on stderr. Maybe the way to go is to just leave it there in all modes.

Relationships

OpenCSW Bug Tracker