A 1983-born man was caught on security cameras at the Summer House of Aspazija and Rainis as he was walking away after he pocketing an inkpot and a dip pen that belonged to Jānis Pliekšāns, best known as Rainis, a seminal figure in Latvian letters.
He was staying at a Rīga hostel where an employee tipped off the police after recognizing the man, the State Police said on Facebook.
The relics, worth an estimated €14,000, will be returned to the museum.
Police say the man had a previous criminal record.