0004579: Can not install in zone

(0008417)
japester (reporter)
2010-10-27 16:33

This is the issue.

dios[1032]~/CSWapache2> grep '/usr/' pkgmap
1 f none /usr/sadm/install/scripts/i.cswap2mod 0555 root bin 2051 27131 1286585397
1 f none /usr/sadm/install/scripts/r.cswap2mod 0555 root bin 1021 10295 1286585397
dios[1033]~/CSWapache2>

The package attempts to install files into areas outside of the scope of /opt/csw/ This will always fail in a sparse root zone.
Reading through the scripts themselves, they should go somewhere like '/opt/csw/apache2/sbin'

(0008418)
bwalton (administrator)
2010-10-27 17:06

Well, the files can't go in apache2/sbin as they're class action scripts. They must live in /usr/sadm/install/scripts. That being said, I'm looking at options to move them to a separate package. The dependencies involved make this not as straight forward as that might sound though.

(0008420)
james (reporter)
2010-10-27 17:42

Then it can't use classaction scripts. They are a pain in CSWcswclassutils as it is, we certainly don't need to add to the problem.

(0008421)
bwalton (administrator)
2010-10-27 17:52

Well, it doesn't specifically need to, but other apache modules will use these. I can split them out so the core apache packages aren't dependent on them. I need to do this carefully though.

I thought we had decided that sparse root wasn't a "supported" target though...?

(0008422)
james (reporter)
2010-10-27 18:03

"we"? sparse zones not supported and I leave the project today.

The only thing that was reluctantly agreed for now was that global zone interaction was needed to install CSWcswclassutils. That still excludes some users and I bet it's not necessary (the use of CAS that is, I say necessary not useful).

(0008423)
japester (reporter)
2010-10-27 18:17

http://www.opencsw.org/use-it/sharing-optcsw/ [^] refers to CSWcswclassutils being special, as it's the only package to install content into /usr.
Do we really need apache2 to be just as special?

Random left wing thought, is it possible to do something similar to what Debian does, with its a2enmod and a2dismod scripts? They let the end user manage enabling/disabling modules, rather than having them automatically done. That way the scripts can be kept inside the /opt/csw/apache directory scope, without having to resort to a classutil script.

(0008458)
bwalton (administrator)
2010-11-16 03:47

This issue should be fully addressed with the packages available at: http://buildfarm.opencsw.org/experimental.html#apache2 [^]

Feedback/testing welcome.

(0008473)
japester (reporter)
2010-11-20 18:01

on a freshly installed clean sparse zone:

# pkg-get -s http://buildfarm.opencsw.org/opencsw/experimental/apache2 [^] install apache2
...
Generating dummy ssl key and certificate...
chmod: WARNING: can't access /opt/csw/apache2/etc/server.key
chmod: WARNING: can't access /opt/csw/apache2/etc/server.crt
chown: /opt/csw/apache2/etc/server.key: No such file or directory
chown: /opt/csw/apache2/etc/server.crt: No such file or directory
pkgadd: ERROR: class action script did not complete successfully
[ verifying class <build> ]
ERROR: attribute verification of </opt/csw/apache2/etc/server.crt.CSW> failed
pathname does not exist
Installing class <cswinitsmf> ...
WARNING! FMRI path contained an illegal dot (removed)
...

Looks like the installation script is failing to create the dummy SSL cert files.
This leaves the package in a 'not completely installed' state, which means that other pkgs which depend on apache won't install (apache2rt, etc)

The FMRI error appears to be taken care of in an updated classutils package (which hasn't been published yet)

Went looking for the postinstall script, but couldn't find it. where is it?

(0008474)
bwalton (administrator)
2010-11-20 19:05

The postinstall script has been removed (in svn, if that's where you're looking) in favour of the build CAS.

Can you verify if /opt/csw/apache2/etc/server.key and /opt/csw/apache2/etc/server.crt get created properly?

I think I see the problem but will wait to hear back.

Thanks
-Ben

(0008475)
japester (reporter)
2010-11-21 15:11

I'm not using the svn version (i used the one as referenced in your comment above).

as mentioned, the ssl keys do NOT get generated at all.

(0008476)
bwalton (administrator)
2010-11-21 15:37

You should be able to see the script used here:
/var/sadm/pkg/CSWapache2/save/build/opt/csw/apache2/etc/server.crt.CSW

That script is executed by the build CAS. It's triggered during the installation of /opt/csw/apache2/etc/server.crt.CSW. The content of that file will be any stdout generated during the run, which should only be the two echo statements. You could look in that file to see if there is a further hint about why the cert files weren't created properly.

What type of zone is this?

Thanks
-Ben

(0008477)
japester (reporter)
2010-11-21 17:01

sparse zone. as already stated.
Just out of interest, why create a dummy ssl cert?

Found the bug. installing openssl in a zone does not create /opt/csw/etc/ssl/openssl.conf, which then makes cert generations fail.

That's a bug in one of the openssl packages.

(0008478)
bwalton (administrator)
2010-11-21 17:12

Ah yes. Sorry about that. I'm recovering from the flu. :(

Ok, that's a good find. It will also be a show stopper for releasing these updates. If you could manually create openssl.conf and test again to ensure that the apache changes themselves are properly valid?

I generate the dummy ssl cert so that httpd-ssl.conf can be enabled after initial install. I guess I could do away with this and leave that disabled in a default install which is likely not all that helpful anyway. Yes, I'm convinced. I'll remove this 'feature' and re-roll.

(0008479)
japester (reporter)
2010-11-21 17:15

with a valid openssl.conf file, the cert creation section does work.

(0008530)
japester (reporter)
2010-12-07 15:06

I note that the experimental builds have disappeared. Are you planning on pushing this into current soon?

(0008531)
bwalton (administrator)
2010-12-07 15:10

I removed them as I broke the update process the other night and didn't want to leave broken packages there. I'll release them when I've nailed down the migration of /opt/csw/apache2/etc to /etc/opt/csw/apache2.

Hopefully will get to work on this tonight.

(0008545)
bwalton (administrator)
2010-12-09 03:23

The packages in the experimental apache2 repo address this.

Please test.

(0008551)
japester (reporter)
2010-12-11 06:34

Appears to upgrade cleanly :)

(0008603)
japester (reporter)
2010-12-19 14:56

eeep!
something that we need to be aware of with the change to /etc/opt/csw/apache, we have to update all (sigh, yes, all) apache2 modules that drop a config file into the apache/extra directory.

(0008604)
bwalton (administrator)
2010-12-19 15:04

Yes, I'm looking at this too. It's a really nasty migration overall. My current thinking is to copy _all_ files from old directory to new and then run a few regex replacements on everything.

I'm sure there are still some corner cases that will sneak through, but...

(0008726)
bwalton (administrator)
2011-01-22 15:40

This should be addressed in the packages at: http://buildfarm.opencsw.org/experimental.html#apache2 [^]

Modules are now handled with the build CAS instead of a custom written one.

Please test.

Thanks
-Ben

(0008736)
james (reporter)
2011-01-24 14:30

Don't want to get too deep in to debugging but here are some quick observations.

start fails complaining of missing /opt/csw/apache2/var/run/, this used to be in apache2c.

after making the directory and trying to start manually (easier to see output):

/opt/csw/apache2/sbin/apachectl -k start
/opt/csw/apache2/sbin/apachectl: /opt/csw/apache2/sbin/httpd: not found

which isn't surprising because it's not there. Copy httpd.worker to httpd, start and it runs.

(0008738)
bwalton (administrator)
2011-01-24 18:23

> start fails complaining of missing /opt/csw/apache2/var/run/, this used to be
> in apache2c.

Ok, I've adjusted the postinstall script to create this on demand rather than making it owned by the package. I'm repackaging now to include this change.

> /opt/csw/apache2/sbin/apachectl -k start
> /opt/csw/apache2/sbin/apachectl: /opt/csw/apache2/sbin/httpd: not found
>
> which isn't surprising because it's not there. Copy httpd.worker to httpd,
> start and it runs.

Again, this should be handled by the cswalternatives CAS. Do you have CSWalternatives installed?

What does the output of: alternatives --display httpd
show?

Mine is:

root @ m2 : ~
# alternatives --display httpd
Installed alternatives for httpd are:
/opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.worker 100
/opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.prefork 50

If you have the CSWalternatives package installed and your output after updating is different than the above (assuming you've added ap2_worker), I'd like to see that output.

Also, are you using pkg-get still, with the broken remove/add sequence? Are you updating from 2.2.13 where the symlink handling of httpd didn't work correctly in some removal scenarios?

Thanks
-Ben

(0008739)
james (reporter)
2011-01-24 20:06

# alternatives --display httpd
Installed alternatives for httpd are:
/opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.worker 100
/opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.prefork 50

ap2_worker was previously installed and is updated during general update.

Suspect part of update:
Installing ap2_worker - The apache worker mpm. as <CSWap2worker>

## Installing part 1 of 1.
/opt/csw/apache2/sbin/httpd.worker
/opt/csw/share/doc/ap2_worker/license
[ verifying class <none> ]
Registering 'httpd' alternative /opt/csw/apache2/sbin/httpd.worker ...
ERROR: /opt/csw/apache2/sbin/httpd exists, and is not symlink.
Not installing /opt/csw/apache2/sbin/httpd.worker as /opt/csw/apache2/sbin/httpd
[ verifying class <cswalternatives> ]

pkg-get does not use a broken sequence unless the dependencies of the packages define a broken sequence. Previously the superiority of the worker package to apache mattered, I'm not sure what is needed now as it uses alternatives.

I'm updating from the newest working package, that is 2.2.13. It's not possible to update from the newer version, see bug 0004579...

(0008951)
bwalton (administrator)
2011-04-01 22:33

This is resolved in the 2.2.17 release.

Notes
(0008417) japester (reporter) 2010-10-27 16:33	This is the issue. dios[1032]~/CSWapache2> grep '/usr/' pkgmap 1 f none /usr/sadm/install/scripts/i.cswap2mod 0555 root bin 2051 27131 1286585397 1 f none /usr/sadm/install/scripts/r.cswap2mod 0555 root bin 1021 10295 1286585397 dios[1033]~/CSWapache2> The package attempts to install files into areas outside of the scope of /opt/csw/ This will always fail in a sparse root zone. Reading through the scripts themselves, they should go somewhere like '/opt/csw/apache2/sbin'

(0008418) bwalton (administrator) 2010-10-27 17:06	Well, the files can't go in apache2/sbin as they're class action scripts. They must live in /usr/sadm/install/scripts. That being said, I'm looking at options to move them to a separate package. The dependencies involved make this not as straight forward as that might sound though.

(0008420) james (reporter) 2010-10-27 17:42	Then it can't use classaction scripts. They are a pain in CSWcswclassutils as it is, we certainly don't need to add to the problem.

(0008421) bwalton (administrator) 2010-10-27 17:52	Well, it doesn't specifically need to, but other apache modules will use these. I can split them out so the core apache packages aren't dependent on them. I need to do this carefully though. I thought we had decided that sparse root wasn't a "supported" target though...?

(0008422) james (reporter) 2010-10-27 18:03	"we"? sparse zones not supported and I leave the project today. The only thing that was reluctantly agreed for now was that global zone interaction was needed to install CSWcswclassutils. That still excludes some users and I bet it's not necessary (the use of CAS that is, I say necessary not useful).

(0008423) japester (reporter) 2010-10-27 18:17	http://www.opencsw.org/use-it/sharing-optcsw/ [^] refers to CSWcswclassutils being special, as it's the only package to install content into /usr. Do we really need apache2 to be just as special? Random left wing thought, is it possible to do something similar to what Debian does, with its a2enmod and a2dismod scripts? They let the end user manage enabling/disabling modules, rather than having them automatically done. That way the scripts can be kept inside the /opt/csw/apache directory scope, without having to resort to a classutil script.

(0008458) bwalton (administrator) 2010-11-16 03:47	This issue should be fully addressed with the packages available at: http://buildfarm.opencsw.org/experimental.html#apache2 [^] Feedback/testing welcome.

(0008473) japester (reporter) 2010-11-20 18:01	on a freshly installed clean sparse zone: # pkg-get -s http://buildfarm.opencsw.org/opencsw/experimental/apache2 [^] install apache2 ... Generating dummy ssl key and certificate... chmod: WARNING: can't access /opt/csw/apache2/etc/server.key chmod: WARNING: can't access /opt/csw/apache2/etc/server.crt chown: /opt/csw/apache2/etc/server.key: No such file or directory chown: /opt/csw/apache2/etc/server.crt: No such file or directory pkgadd: ERROR: class action script did not complete successfully [ verifying class <build> ] ERROR: attribute verification of </opt/csw/apache2/etc/server.crt.CSW> failed pathname does not exist Installing class <cswinitsmf> ... WARNING! FMRI path contained an illegal dot (removed) ... Looks like the installation script is failing to create the dummy SSL cert files. This leaves the package in a 'not completely installed' state, which means that other pkgs which depend on apache won't install (apache2rt, etc) The FMRI error appears to be taken care of in an updated classutils package (which hasn't been published yet) Went looking for the postinstall script, but couldn't find it. where is it?

(0008474) bwalton (administrator) 2010-11-20 19:05	The postinstall script has been removed (in svn, if that's where you're looking) in favour of the build CAS. Can you verify if /opt/csw/apache2/etc/server.key and /opt/csw/apache2/etc/server.crt get created properly? I think I see the problem but will wait to hear back. Thanks -Ben

(0008475) japester (reporter) 2010-11-21 15:11	I'm not using the svn version (i used the one as referenced in your comment above). as mentioned, the ssl keys do NOT get generated at all.

(0008476) bwalton (administrator) 2010-11-21 15:37	You should be able to see the script used here: /var/sadm/pkg/CSWapache2/save/build/opt/csw/apache2/etc/server.crt.CSW That script is executed by the build CAS. It's triggered during the installation of /opt/csw/apache2/etc/server.crt.CSW. The content of that file will be any stdout generated during the run, which should only be the two echo statements. You could look in that file to see if there is a further hint about why the cert files weren't created properly. What type of zone is this? Thanks -Ben

(0008477) japester (reporter) 2010-11-21 17:01	sparse zone. as already stated. Just out of interest, why create a dummy ssl cert? Found the bug. installing openssl in a zone does not create /opt/csw/etc/ssl/openssl.conf, which then makes cert generations fail. That's a bug in one of the openssl packages.

(0008478) bwalton (administrator) 2010-11-21 17:12	Ah yes. Sorry about that. I'm recovering from the flu. :( Ok, that's a good find. It will also be a show stopper for releasing these updates. If you could manually create openssl.conf and test again to ensure that the apache changes themselves are properly valid? I generate the dummy ssl cert so that httpd-ssl.conf can be enabled after initial install. I guess I could do away with this and leave that disabled in a default install which is likely not all that helpful anyway. Yes, I'm convinced. I'll remove this 'feature' and re-roll.

(0008479) japester (reporter) 2010-11-21 17:15	with a valid openssl.conf file, the cert creation section does work.

(0008530) japester (reporter) 2010-12-07 15:06	I note that the experimental builds have disappeared. Are you planning on pushing this into current soon?

(0008531) bwalton (administrator) 2010-12-07 15:10	I removed them as I broke the update process the other night and didn't want to leave broken packages there. I'll release them when I've nailed down the migration of /opt/csw/apache2/etc to /etc/opt/csw/apache2. Hopefully will get to work on this tonight.

(0008545) bwalton (administrator) 2010-12-09 03:23	The packages in the experimental apache2 repo address this. Please test.

(0008551) japester (reporter) 2010-12-11 06:34	Appears to upgrade cleanly :)

(0008603) japester (reporter) 2010-12-19 14:56	eeep! something that we need to be aware of with the change to /etc/opt/csw/apache, we have to update all (sigh, yes, all) apache2 modules that drop a config file into the apache/extra directory.

(0008604) bwalton (administrator) 2010-12-19 15:04	Yes, I'm looking at this too. It's a really nasty migration overall. My current thinking is to copy _all_ files from old directory to new and then run a few regex replacements on everything. I'm sure there are still some corner cases that will sneak through, but...

(0008726) bwalton (administrator) 2011-01-22 15:40	This should be addressed in the packages at: http://buildfarm.opencsw.org/experimental.html#apache2 [^] Modules are now handled with the build CAS instead of a custom written one. Please test. Thanks -Ben

(0008736) james (reporter) 2011-01-24 14:30	Don't want to get too deep in to debugging but here are some quick observations. start fails complaining of missing /opt/csw/apache2/var/run/, this used to be in apache2c. after making the directory and trying to start manually (easier to see output): /opt/csw/apache2/sbin/apachectl -k start /opt/csw/apache2/sbin/apachectl: /opt/csw/apache2/sbin/httpd: not found which isn't surprising because it's not there. Copy httpd.worker to httpd, start and it runs.

(0008738) bwalton (administrator) 2011-01-24 18:23	> start fails complaining of missing /opt/csw/apache2/var/run/, this used to be > in apache2c. Ok, I've adjusted the postinstall script to create this on demand rather than making it owned by the package. I'm repackaging now to include this change. > /opt/csw/apache2/sbin/apachectl -k start > /opt/csw/apache2/sbin/apachectl: /opt/csw/apache2/sbin/httpd: not found > > which isn't surprising because it's not there. Copy httpd.worker to httpd, > start and it runs. Again, this should be handled by the cswalternatives CAS. Do you have CSWalternatives installed? What does the output of: alternatives --display httpd show? Mine is: root @ m2 : ~ # alternatives --display httpd Installed alternatives for httpd are: /opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.worker 100 /opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.prefork 50 If you have the CSWalternatives package installed and your output after updating is different than the above (assuming you've added ap2_worker), I'd like to see that output. Also, are you using pkg-get still, with the broken remove/add sequence? Are you updating from 2.2.13 where the symlink handling of httpd didn't work correctly in some removal scenarios? Thanks -Ben

(0008739) james (reporter) 2011-01-24 20:06	# alternatives --display httpd Installed alternatives for httpd are: /opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.worker 100 /opt/csw/apache2/sbin/httpd httpd /opt/csw/apache2/sbin/httpd.prefork 50 ap2_worker was previously installed and is updated during general update. Suspect part of update: Installing ap2_worker - The apache worker mpm. as <CSWap2worker> ## Installing part 1 of 1. /opt/csw/apache2/sbin/httpd.worker /opt/csw/share/doc/ap2_worker/license [ verifying class <none> ] Registering 'httpd' alternative /opt/csw/apache2/sbin/httpd.worker ... ERROR: /opt/csw/apache2/sbin/httpd exists, and is not symlink. Not installing /opt/csw/apache2/sbin/httpd.worker as /opt/csw/apache2/sbin/httpd [ verifying class <cswalternatives> ] pkg-get does not use a broken sequence unless the dependencies of the packages define a broken sequence. Previously the superiority of the worker package to apache mattered, I'm not sure what is needed now as it uses alternatives. I'm updating from the newest working package, that is 2.2.13. It's not possible to update from the newer version, see bug 0004579...

(0008951) bwalton (administrator) 2011-04-01 22:33	This is resolved in the 2.2.17 release.

Relationships

OpenCSW Bug Tracker