They were judged inadequate twice.
It was the second monitoring visit since the council’s children’s services were declared inadequate for a second time last June.
The visit by two inspectors in April followed “serious and widespread child protection concerns” identified in January.
A report after the April visit said: “The local authority is starting to make some progress, from an extremely low base, in improving services for its children and young people, but these improvements remain exceedingly fragile.”
Torbay council’s director of children’s services Alison Botham welcomed the recognition by inspectors of the hard work and commitment by staff and managers, and added: “I know that there is still much to be done.”
The report said the council found it difficult to recruit enough experienced social workers and caseloads were still too high.
That meant despite working extremely hard staff were “unable to intervene effectively with many children in need of help and protection.”
The result was the quality of practice for a significant number of children was “extremely concerning.”
But the inspectors found “good examples of effective multi-agency assessments and interventions that are making a real difference to children.”
The report added: “A significant strength in Torbay is the commitment of staff and managers to do the right thing for children.
“Most social workers told inspectors that despite the high levels of work, they like working in Torbay and feel well supported.”
The report said extra staff had been brought in and the foundations were in place for medium and longer-term improvements.
The report said: “Leaders are aware that practice is not consistently safe enough for too many children in Torbay and are working diligently to address the significant deficits.
“Caseloads are still too high. Leaders accept that the pace of change needs to accelerate.”
The report said recruitment and retention of staff was a “priority and huge challenge”.
The service currently relied on high numbers of agency staff and senior leaders showed a “relentless commitment” to workforce development and training.
Ms Botham said: “I am pleased that the inspectors recognised the continued hard work and commitment of the staff and managers in Torbay’s Children’s Social Care services, and that they were able to highlight some examples of good practice.
“We share the inspectors’ views about the impact for children of the variability in practice and I know that there is still much to be done.
“High caseloads have impacted upon social workers’ ability to deliver consistent improvement.
“The inspectors recognised the actions we are taking to provide the capacity and support for all staff to deliver their best practice.
“This includes significant additional social work capacity for the next 12 months to enable longer term solutions to be agreed and implemented.
“It is a real credit to all staff and managers working in Torbay that their commitment to doing the right thing for children remains undaunted and this was acknowledged as a significant strength by Ofsted.”