Resetting root Password in Solaris

If you don't want the life story that goes along with this article, just use these links to access the relevant instruction set:

At my office we received a couple Sunfire T1000 servers for internal use as application servers. We had a consultant come into the office to setup the software on these servers, but it was a few months after they left before we started to be ready to use the application. I was tasked with finalizing the setup and moving the servers onto the proper network. I was not really a part of this application setup at all prior to this so I knew very little about it. I figured it would be fairly simple though, it's Solaris, which is close enough to Linux that I could find my way around and make the configuration changes necessary and be done with it.

So, I go get the servers, push the button on the front to power them down and move them to the new data center and get them racked up. I go to get hooked up to them so I can login and change the settings and realize there is no keyboard, monitor, or mouse port on these Sunfires. How do I get into them? Well, I know they have ssh going so I try that, I get the login for the servers from the manager and SSH in first to the non-superuser account, then try and su up so I can make the configuration changes. But what's this? The root password does not work? Uh-oh, gotta reset it.

So I do some research online, everyone talks about just booting from CD into single user mode and editing the shadow file to remove the password. Ok, that would be great if these servers had a CD drive. I tried just booting into single user mode without a CD but it requires the password before it lets you in, so that was no help. I ask the manager to get a hold of the consultant we had hired a few months back to try and get the password, while I research some other methods of getting access. I find out about this whole jump start server process and doing network boots but try as I might I just could not figure out how to get a jump start server going.

The next day we get an email back from the consultant and he emails us his document with the password. Hooray, we can get in now. I make sure I save that password and login to the server, make the changes necessary and bring it up. Great, one down, one to go. I power up the second server, try and SSH into it just like before, but am denied. Great, apparently the non-super user account is either non-existent or is something different. Either way, nobody knows it so we are screwed again. So I need to figure out how to console into this box. I do some more research on these sun servers and see the management port, both for console access with a console cable and one for SSH access. I have no console cable, so SSH it is I decided.

I get everything setup, connect to the ssh management port, and get presented with a password prompt. I give the root login a try, does not work. I try some likely default stuff, does not work. sigh Apparently the ALOM has it's own password configured and it needs to be reset as well, which means I need to console in. I manage to dig up a Console cable for Cisco routers, and barrow a Serial to USB converter. I get console in, but have the same password prompt and can't do anything. I do some research on how to reset the ALOM password, and get it figured out. It's actually pretty easy to do, here are the steps:

Resetting ALOM password

  1. Power down the system and unplug the power cable. Wait a little bit for everything to get turned off.
  2. Connect the console cable to the management port, and setup the connection so you can see the ALOM booting up. Reconnect the power cable to begin the ALOM boot up
  3. Press the ESC key during boot-up to get the ALOM menu.
  4. This will present you with a menu, and one of the options will be to reset the NVRAM. Select this option.
  5. Once the NVRAM is reset, sc will be back to it's default config with no password. Before you can use SC to power on the machine you will be required to set the password.

Yay, the ALOM password has been reset. I decide I better check the working box and see if I need to do the same thing. Turns out I do so I repeat the steps above and reset it's ALOM password.

Now that I have console access, I try logging into the Solaris install as root, but of course that does not work, it could not be that easy for me. I have to reset this boxes root password now. Since I never actually figured out how to do it with the last one (got lucky and got the password from the consultant) I now am back to square one on this one. I try quite a few things, nothing really worked.

I tried putting the hard drive into the other sun box, booting it up, and using it as a host system to reset the password. The box did not recognize the hard drive though so I was not able to do anything. It turns out all I needed to do was boot with the -r flag and it would have been seen, but being the Solaris/SUN newbie I am I did not find out about that until about two weeks after the fact. For anyone else in a similar situation, here are the steps you should use if you can:

Using the proper method

  1. Power down the solaris box you need to reset the password on, and remove the hard drive.
  2. Power down a second solaris box to add the drive to. Connect the drive as a secondary drive.
  3. Power on the box with both hard drives. The T1000's have a kill switch so if the cover is off it won't power on. If your box is the same, you will need to hold this switch down, or replace the cover. In my case, the cover wont fit with both drives in, so I held the switch down with a pair of siscors. The switch is in the front left corner.
  4. You need to halt the boot process during boot up by using the ALOM and it's break command. This will get you to the Open Boot 'ok ' prompt.
  5. At the ok prompt, issue the command boot -r to boot the machine. The -r flag tells solaris to reconfigure it's hardware profile so it recgonizes the second hard drive.
  6. Once booted, mount the secondary drive on a directory somewhere. Mount the partition containing /etc/shadow.
  7. Edit the shadow file on the drive and either remove the encrypted password string (between the two colons after the username 'root') or replace it with a known encryption string. In my case, I replaced it with the string for the other server so their passwords are the same.
  8. Shutdown the server, replace the drives, and boot them both back up again. I booted with the -r flag again so the server recgonized it only had one drive again.

Since I did not know about that way of doing the reset at the time, I ended up finding another solution after a little more trial and error. I knew Solaris used UFS partitions and I knew that the linux kernel had a UFS driver. Therefore I built a linux install on a spare hard drive, and compiled the UFS driver into the kernel with write support. This was all in vein though as it turns out the write support is only for an older version of UFS and does not work with Solaris.

My next attempt was to install Solaris x86 and try to use that to edit the hard drive. I figured it's Solaris, it uses UFS too, it should work. It turns out though, that Solaris x86 cannot mount or work with a UFS partition that was created using Solaris SPARC. Damn those platform differences.

What I finally did that worked, was I went back to my linux install and used the dd command to do direct reading and writing from and to the hard drive. I read from the hard drive, adjusting the seek/count parameter until I found the string in the shadow file I wanted to replace. I then did the same thing on the working sun box in order to read the encrypted string from it's shadow file. I then used dd to write the string to the first drive's shadow file. Once written I put the drive back in the box, and crossed my fingers hoping it would still boot up, and that I could log in. By some miracle it did, it all worked and now I finally had both boxes setup with known passwords in all areas so I could get some work done. This whole experimenting in trying to reset these passwords took me about a week to complete. Here is a brief step-by-step of how to use dd to reset the password:

Using the hack method

  1. Either build up a linux box, or just grab a copy of knoppix or some other live CD to use.
  2. Connect the hard drive to the Linux box as a secondary hard drive.
  3. Use the dd command to read from the drive. Set the if parameter to the partition of the drive with the shadow file on it. Set the bs parameter to 1M to read 1 meg at a time. Set the count parameter to 1000, and set the of parameter to send the output to a file. Make sure you have sufficient space (1gig) to save this section of data, or else modify the parameters to something you can accommodate. An example of the command is:

    root@localhost:~#dd if=/dev/hdb1 of=hdb1.img bs=1M count=1000
  4. When the data is read and saved, use the grep command with the -ab flag to search for root:.

  5. If something is found, the byte offset and context of it in the file will be printed. If not, repeat step 3, but use the skip parameter to skip over what you just searched. Example:

    root@localhost:~#dd if=/dev/hdb1 of=hdb1.img bs=1M count=1000 skip=1000
  6. Make sure that the context looks like what the shadow file should. If not, repeat step 3, using skip to move ahead.

  7. Once you have found the byte position of the root: marker, calculate the true offset based on the bs, count, and skip parameters. Example, if you found it with the command of:

    root@localhost:~#dd if=/dev/hdb1 of=hdb1.img bs=1M count=1000 skip=3000

    and the byte offset was 1729381 then the offset inside /dev/hdb1 would truly be: (1024*1048)*1000*3000+1729381=321,945,772,9381

  8. Use the new offset to pull the file, first we pull a fair amount to ensure it is the correct offset, we may need to tweak it with trial and error.

    root@localhost:~#dd if=/dev/hdb1 of=hdb1.img bs=1 count=50 skip=3219457729381
  9. Once you found the right spot, adjust the skip parameter so it starts right after the : after root, and set count to stop right before the colon after the password. In my case, count ended up being 13.

  10. Create a text file which contains an encrypted version of a password you do know. The encrypted string has to be the same length as the existing string.
  11. Use dd to overwrite the string on the hard drive with the string on the file. Change the skip parameter to seek, make of the hd, and if the text file containing the new password. Example:

    root@localhost:~#dd if=newpass.txt of=/dev/hdb1 bs=1 count=13 seek=3219457729381

That is the gist of how to use dd to reset the password. Be very careful if you want to use this method, as if you mess up the command line you could potentially wipe your hard drive not be able to recover it.

Hopefully this little story will help anyone who might end up in a situation like mine, or maybe worse. The last method is the only way I can think of right now to reset the password if you had no spare SPARC box to boot from, and all you had was x86 boxes.