Anti-Defamation League (ADL) have claimed that the Twitter owner's decision to reinstate the account of former US President Donald Trump is "dangerous and inconsistent".
The group adds that allowing rapper Kanye West back on the platform has "raised concerns".
Twitter have not responded to requests for comment but the company says it has "long-standing fruitful relationships with key civil rights groups and organisations, including NAACP, ADColor, and the Anti-Defamation League".
According to the ADL, Musk met with the civil rights group on November 1 to assure them that nobody banned from the platform would be reinstated until he had installed a transparent process that took into consideration the views of society.
ADL CEO Jonathan Greenblatt says that "Mr Musk's decisions over the last month have raised serious concerns".
The tycoon allowed Trump back on Twitter over the weekend after conducting a poll asking users whether or not the politician should be given the chance to return after he was banned following the US Capitol riot last year.
The poll registered over 15 million votes and users narrowly supported the move by 52 per cent to 48 per cent.
Musk tweeted "the people have spoken" and reinstated Trump's account, although the former President hasn't made any posts yet.
Greenblatt said: "This decision is dangerous and inconsistent with what Musk previously had indicated to our group. It forces us to wonder whether he is serious about safeguarding the platform from hate, harassment and disinformation."