The sackings of Chris Powell and Chris Ramsey were "disheartening", says anti-racism campaigner Troy Townsend.