We booked an organized tour for Lofthellir Lava cave that is known to have the greatest natural ice sculptures. It was a 45-minute drive South-East from Lake Mývatn on a regular road then 1 hour on a very bumpy road through the lava fields (the last part was pretty rough, especially if you suffer from motion sickness :s). Thankfully, I was sitting next to the driver, so I didn’t feel that bad :p
The guide gave us helmets with with a light, a pair of boots and we headed down to the cave. The entrance was very narrow and we had to crawl to get into the cave and it was very slippery. Once we got inside, our mind was blown away with all these odd shapes and ice formations. The rooms were huge, more than 6 meters sometimes, it was very impressive.
I would recommend to bring warm clothes (waterproof if possible) and gloves are also a good thing to have. Unless you have a waterproof trousers, you will end up quite wet, so taking some spare clothes with may not be a bad idea :-))