In verse 5 the poor of the Zoramites voice a concern. They complain to Alma that they have no place to worship because the rich have kicked em out of the synagogues. Thinking of my own life, I really only consider going to church on Sunday or going to the temple as qualifying as worship. It's likely that I would feel very much like these Zoramites if all of a sudden I was no longer allowed to enter church buildings or temples.
In the next chapter Alma says in verse 3, "Do ye remember to have read what Zenos, the prophet of old, has said concerning prayer or worship?" He then goes on to explain that praying unto God in the wilderness, over our fields, and in our closets is worshipping God.
Prayer is worship, if we make it that by being sincere and thanking and praising God in our conversation with Him. Richard G. Scott said, "Think of it: the absolute Supreme Being, the most all-knowing, all-seeing, all-powerful personage, encourages you and me, as insignificant as we are, to converse with Him as our Father."