Do we have any other historical data to put this in context? It seems that in the 4th century there was conflict on this issue.
The issue is related to the similar question of
clerical celibacy (link to wikipedia, which has a good survey of quotes and conflicting opinions on the topic). In other words, one reason polygamy often isn't addressed directly, is because the church is more concerned with whether clergy should even be married
at all (and if so, whether they were permitted to have intercourse). At the First Council of Nicea, in 325, there was some discussion about whether clerical celibacy should be included. According to one historian:
While [the bishops at Nicaea] were deliberating about this, some thought that a law ought to be passed enacting that bishops and presbyters, deacons and subdeacons, should hold no intercourse with the wife they had espoused before they entered the priesthood; but Paphnutius, the confessor, stood up and testified against this proposition; he said that marriage was honorable and chaste, and that cohabitation with their own wives was chastity, and advised the Synod not to frame such a law, for it would be difficult to bear, and might serve as an occasion of incontinence to them and their wives; and he reminded them, that according to the ancient tradition of the church, those who were unmarried when they took part in the communion of sacred orders, were required to remain so, but that those who were married, were not to put away their wives. Such was the advice of Paphnutius, although he was himself unmarried, and in accordance with it, the Synod concurred in his counsel, enacted no law about it, but left the matter to the decision of individual judgment, and not to compulsion.
However, they did pass Canon 3 which reads: "The great Synod has stringently forbidden any bishop, presbyter, deacon, or any one of the clergy whatever, to have a subintroducta dwelling with him, except only a mother, or sister, or aunt, or such persons only as are beyond all suspicion." A "subintroducta" appears to refer to an unmarried woman, living in the same dwelling (apparently, this was a thing that happened). That single Christian women devoted to God could no longer live with the clergy was undoubtedly one factor that eventually led to the establishment of nunneries.