0003831: Saving of admin file still doesn't work correctly

(0006522)
dam (administrator)
2009-08-07 12:50

I did start another instance of pkgutil in parallel to add a missing package. May this have deleted the admin-file? It should then be annoted with a trailing pid or something.

(0006523)
bonivart (developer)
2009-08-07 15:49

I think (hope) that it was the second instance that caused the problem. Maybe some kind of lock system is necessary?

(0006524)
dam (administrator)
2009-08-07 15:53

You may want to look at pca, which is also written in Perl and does fine-grained locking. For now copying admin to admin.$$ would be sufficient for this problem, but you need to also fix multiple concurrent downloads of the same file (which is handled in pca).

(0006571)
bonivart (developer)
2009-08-13 17:47

For the options that might need the admin file I copy admin to admin.run already, this is to make it work during an upgrade of itself, it's admin.run that is used during pkgadd/pkgrm operations. I then remove admin.run before pkgutil exits so I think just checking for its existence may be enough to stop multiple instances from screwing up for each other. Of course simple options like help, version, findfile, listfile and so on are not affected.

I made the message include that a file (normally /var/opt/csw/pkgutil/admin.run) was found where none should be so if the user knows there are no other instances (something was left from an aborted run?) it can easily be removed.

Is that good enough for now?

What would the "multiple concurrent downloads of the same file" be used for?

(0006575)
bonivart (developer)
2009-08-14 19:25

I just noticed that pkgutil leaves the admin.run file lots of times which previously was no problem since it was just overwritten but now when I have the "multiple instance"-protection it's a problem. One thing is to clean up better before exiting but an unexpected (e.g. user aborted) exit can always happen. I have therefor also added checks for other pkgutil processes when an admin.run file is found, if only one process is running the file is considered stale and removed which should work in most cases. This is in r80.

(0006585)
dam (administrator)
2009-08-17 16:40

When download is ran as a normal user the admin-file can't be written:

build8st% id
uid=10000(dam) gid=10000(csw)
build8st% pkgutil -t http://mirror.opencsw.org/opencsw/testing [^] -d perl
Can't open /var/opt/csw/pkgutil/admin.run: Permission denied

Maybe you should write the lockfile to /var/run, additionally, on downloading there should be no necessity for locking at all.

(0006586)
bonivart (developer)
2009-08-18 10:36

Sure it's a bug but it wouldn't have worked anyway since /var/opt/csw/pkgutil is only writable as root so the packages couldn't have been written there either. From the start pkgutil was only meant to be run as root and then I think it was you who requested it to be possible to use some things as non-root and -W (workdir) came out of it. Can you try if that works?

(0006692)
bonivart (developer)
2009-09-10 13:31

Have you had a chance to try the 1.7 beta?

(0006764)
bonivart (developer)
2009-09-28 15:35

Fix is in 1.7. Unable to verify if it's still a problem.

Notes
(0006522) dam (administrator) 2009-08-07 12:50	I did start another instance of pkgutil in parallel to add a missing package. May this have deleted the admin-file? It should then be annoted with a trailing pid or something.

(0006523) bonivart (developer) 2009-08-07 15:49	I think (hope) that it was the second instance that caused the problem. Maybe some kind of lock system is necessary?

(0006524) dam (administrator) 2009-08-07 15:53	You may want to look at pca, which is also written in Perl and does fine-grained locking. For now copying admin to admin.$$ would be sufficient for this problem, but you need to also fix multiple concurrent downloads of the same file (which is handled in pca).

(0006571) bonivart (developer) 2009-08-13 17:47	For the options that might need the admin file I copy admin to admin.run already, this is to make it work during an upgrade of itself, it's admin.run that is used during pkgadd/pkgrm operations. I then remove admin.run before pkgutil exits so I think just checking for its existence may be enough to stop multiple instances from screwing up for each other. Of course simple options like help, version, findfile, listfile and so on are not affected. I made the message include that a file (normally /var/opt/csw/pkgutil/admin.run) was found where none should be so if the user knows there are no other instances (something was left from an aborted run?) it can easily be removed. Is that good enough for now? What would the "multiple concurrent downloads of the same file" be used for?

(0006575) bonivart (developer) 2009-08-14 19:25	I just noticed that pkgutil leaves the admin.run file lots of times which previously was no problem since it was just overwritten but now when I have the "multiple instance"-protection it's a problem. One thing is to clean up better before exiting but an unexpected (e.g. user aborted) exit can always happen. I have therefor also added checks for other pkgutil processes when an admin.run file is found, if only one process is running the file is considered stale and removed which should work in most cases. This is in r80.

(0006585) dam (administrator) 2009-08-17 16:40	When download is ran as a normal user the admin-file can't be written: build8st% id uid=10000(dam) gid=10000(csw) build8st% pkgutil -t http://mirror.opencsw.org/opencsw/testing [^] -d perl Can't open /var/opt/csw/pkgutil/admin.run: Permission denied Maybe you should write the lockfile to /var/run, additionally, on downloading there should be no necessity for locking at all.

(0006586) bonivart (developer) 2009-08-18 10:36	Sure it's a bug but it wouldn't have worked anyway since /var/opt/csw/pkgutil is only writable as root so the packages couldn't have been written there either. From the start pkgutil was only meant to be run as root and then I think it was you who requested it to be possible to use some things as non-root and -W (workdir) came out of it. Can you try if that works?

(0006692) bonivart (developer) 2009-09-10 13:31	Have you had a chance to try the 1.7 beta?

(0006764) bonivart (developer) 2009-09-28 15:35	Fix is in 1.7. Unable to verify if it's still a problem.

Relationships

OpenCSW Bug Tracker