Saint Hripsime Church is nearby the Etchmiadzin Holy See and was built in 618 AD. Armenia was the first nation to convert to Christianity, predating even Rome, and has some of the oldest Christian churches in the world. This church is named for Saint Hripsime, who fled from the Roman Emperor Diocletian's court when he made unwelcome advances towards her only to be tortured and killed by King Tiridates III upon reaching Armenia.
If you're visiting Yerevan in the summer you'll experience sweltering heat during the day. No matter how thirsty you get, resist the urge to drink from any of the public water fountains around Yerevan. You will see many locals crowded around drinking from them but they have built up an immunity to the pathogens in the water. The public drinking fountains are not drawing from the municipal water supply but are pumped in untreated from other sources, so don't think you can drink from them even if you have been able to tolerate the tap water because it is not the same. I'm not entirely clear why the drinking fountains aren't using the same tap water supply, but it is something to keep in mind since you are almost guaranteed to contract a giardia infection from them. Don't do it.
Khachkars are a form of art unique to Armenia. Newcomers may confuse them for gravestones as they tend to be arrayed outside of churches and monasteries but they are actually religious monuments. Khachkars follow a general design pattern depicting the mortal Earth at the bottom, the Church and Christ in the middle, and Heaven at the top. They are symbolic of acceptance of Christ leading man towards Heaven. Echmiadzin hosts a number of intricately carved examples and you'll find many at monasteries around the country such as Novarank and Sevanavank.