An algebraic approach to the theory of regular languages of finite and infinite words (∞-languages) is presented. It extends the algebraic theory of regular languages of finite words, which is based on finite semigroups. Their role is taken over by a structure called right binoid. A variety theorem is proved: there is a one-to-one correspondence between varieties of ∞-languages and pseudovarieties of right binoids. The class of locally threshold testable languages and several natural subclasses (such as the class of locally testable languages) as well as classes of the Borel hierarchy over the Cantor space (restricted to regular languages) are investigated as examples for varieties of ∞-languages. The corresponding pseudovarieties of right binoids are characterized and in some cases defining equations are derived. The connections with the algebraic description and classification of regular languages of infinite words in terms of finite semigroups are pointed out.