Scattered locations throughout the universe
Soul Gem: Adam Warlock (formerly) High Evolutionary, In-Betweener, Magus;
Power Gem: None (formerly) Fallen One, Titania, Drax the Destroyer, Stranger, Magus;
Time Gem: None (formerly) Gardener, Gamora, Magus;
Space Gem: None (formerly) Runner, Pip the Troll, Magus;
Reality Gem: None (formerly) Thanos, Collector;
Mind Gem: None (formerly) Grandmaster, Moondragon, Supreme Intelligence, Magus;
Ego Gem: None (formerly) Loki;
Infinity Gauntlet: (formerly) Thanos, Adam Warlock, Nebula, Man-Beast
The Infinity Gems (previously known as "Soul Gems") are some of the most powerful artifacts in the Universe. Each one allows its owner complete omnipotence over one aspect of the universe: time, space, reality, mind, power, and soul. When all six are directly wielded together, nothing is impossible for their bearer.
The Gems were actually once part of the primordial entity Nemesis, who, eons ago, ended its existence rather than live as the only sentience in the universe. However, the gems were not known to be a collective power for ages. In particular, Adam Warlock bore the Soul Gem and only knew its power over the human mind and its self-contained pocket universe. Warlock was given the Gem by the High Evolutionary, who had obtained it and other Gems under unrevealed circumstances, and he wore it for many years. The Time and Power Gems were owned by cosmic entities the Gardener and the Stranger, respectively.
At some point, Thanos acquired these two and three others-- from an alien prison satellite, a cavern on Deneb IV, and the grip of the "monster" Xiambor -- and scanned Warlock's Soul Gem. Thanos used elements of each of them to create a synthetic "Star Gem." Using a fraction of the Gems' true might, he caused supernovas in an attempt to please Mistress Death, the personification of death itself. Eventually, Thanos was killed, Adam Warlock found peace within the Soul Gem, and the Gems were scattered.
The Elders of the Universe recovered five Gems, and had the Silver Surfer steal the sixth from the Kree Supreme Intelligence. They used the Gems' combined might against Galactus in a bid to replace him in the universe's hierarchy. Reed and Susan Richards used the Gems to foil this scheme, but the Gems were subsequently claimed by five Elders and the cosmic In-Betweener. Soon after this, Mistress Death resurrected her faithful servant Thanos to right what she saw as a great cosmic imbalance -- that more people were alive than had ever died. Using guile and force, Thanos united all Gems for the first time. Wearing them atop his glove as the Infinity Gauntlet, Thanos gained virtual omnipotence. It took an assemblage of heroes and cosmic entities to confront Thanos, now a mad god and spurned by Death, until Warlock could usurp the Gauntlet for himself, correcting all the evil Thanos had done.
Adam Warlock was ruled mentally unfit to be supreme being by the Living Tribunal, and was ordered to find guardians for the Gems. It was also ruled that the Gems could never be used in unison again. Adam then formed his "Infinity Watch," giving the Mind, Space, Power, and Time Gems to Moondragon, Pip the Troll, Drax the Destroyer and Gamora while keeping the Soul Gem for himself. The Reality Gem was secretly given to Thanos, who knew the danger of trying to use it without the others.
The Man-Beast briefly stole the Gems and used them indirectly to create the Infinity Thrall. He was defeated, but another menace soon surfaced: Magus, Warlock's evil side, collected five of the six Gems in his "Infinity War." Thanos and Warlock defeated him by giving him a fake Reality Gem, distracting him long enough for Adam to seize the other Gems.
The Infinity Gems were sought by numerous other beings, including Count Abyss, but were eventually taken by an energy vampire from an alternate-reality. In that other multiverse, they were collected by Loki but then reunited with a seventh gem, the Ego Gem, the remnant of Nemesis' consciousness, who wreaked havoc in both dimensions until destroyed by the Black Knight. Warlock returned from that other dimension bearing the Soul Gem once more.
Returned to reality, the six primary Gems have been briefly re-gathered by Galactus and the Grandmaster. Warlock retains the Soul Gem, the Champion (now, the Fallen One) had briefly given his Gem to Titania to use in a ploy against She-Hulk, and Pip the Troll maintains the Space Gem-granted ability to teleport. The current locations of the other Gems are unknown.