DEPARTMENT OF EDUCATION

OFFICE OF THE INSPECTOR GENERAL

Audit of Monitoring Performed
By the
School Transportation Management Section

Audit Number
08/09-01A

February 20, 2009
Executive Summary
The objectives of this audit were to: determine if transportation program monitoring is adequate based on statutes, rules and other guidance, and evaluate the transportation program monitoring performed by the School Transportation Management Section (STMS). 
Overall, we noted that the STMS is diligent in performing its role and has generally complied with statutes, rules, and other guidance.  We noted four areas in which additional management attention could result in closer compliance with authoritative criteria and good management practice:  
•	An accurate listing of schools is needed to determine the population to be monitored.
•	More complete record keeping is needed.
•	Special Needs requirements were not followed by the school districts.
•	Some districts did not follow documentation directions or did not comply with rules.
We evaluated existing internal controls and other than issues noted in this report found them adequate.

Background
This audit was identified in the Office of Inspector General’s annual risk assessment and included in the approved annual audit plan.  It was performed in support of the Department of Education’s (DOE) goal of quality efficient services, as well as the strategic imperative to align resources to strategic goals.
The School Transportation Management Section (STMS) provides leadership, assistance, and training for school districts and other clients to ensure safe and efficient transportation of Florida's public school students.  This section is also responsible for monitoring school district transportation programs for compliance with laws, rules, and industry campaigns pertaining to safety.
We reviewed monitoring documentation for 20 of the state's 67 school districts.  Districts submit a self-evaluation package on a four-year rotation for review by the STMS.  Our judgmental sample was based on at least four districts from each of the four years in a review cycle and also on the varying sizes of districts’ bus fleets.
Scope and Objectives
The objectives of this audit were to: determine if transportation program monitoring is adequate based on statutes and other guidance, and evaluate the transportation program monitoring performed by the School Transportation Management Section.  The scope of the audit included school years 2005-06 through 2008-09.
Methodology
To achieve the objectives, the auditor: 1) researched and reviewed applicable statutes, rules, policies and procedures, previous audit reports, and relevant information security documentation; 2) interviewed appropriate staff; 3) reviewed and evaluated current authoritative literature; 4) reviewed school district self-monitoring documentation; 5) reviewed STMS monitoring documentation; and 6) evaluated internal controls.
Standards
This audit was conducted in accordance with the International Standards for the Professional Practice of Internal Auditing.

Audit Results
Overall, we noted that the STMS is diligent in performing its role and has generally complied with statutes, rules, and other guidance.  We noted four areas in which additional management attention could result in improved compliance with authoritative criteria and good management practice:  
•	An accurate listing of schools is needed to determine the population to be monitored.
•	More complete record keeping is needed.
•	Special Needs requirements were not followed by the school districts.
•	Some districts did not follow documentation directions or did not comply with rules.
The findings following provide an explanation of the issues.  Each of these findings is organized into six segments:
•	Condition – The factual evidence found in the course of the audit.
•	Criteria – The standards, measures, or expectations used in making an evaluation and/or verification.
•	Cause – The reason for the difference between the expected and actual conditions.
•	Effect – The risk or exposure due to the condition being inconsistent with the criteria (the impact of the finding).
•	Recommendation – Suggested corrective action.
•	Management Response – Auditee opinion and statements regarding concurrence or non-concurrence with the finding and corrective action planned by management.

Findings and Recommendations

Finding 1 – An accurate listing of schools is needed to determine the population to be monitored. 

Condition:	Two criteria used in school transportation monitoring, School Bus Evacuation Drills and School Loading Zones for Buses, are based upon obtaining an accurate and complete list of a district’s schools.  Districts are required to report on a school whether it buses students or not.  For a number of reasons, we had difficulty determining the inventory of schools for reporting.  We noted that both the school districts and STMS encountered problems in reporting on all schools.  Both the districts and STMS usually used the list of schools from the districts’ Web sites.  We used DOE’s Master School Identification (MSID) as a basis.  We noted over 100 schools in 11 of the 20 districts we sampled where MSID schools were not reported on by districts.  Conversely, we also noted districts reporting on schools by districts that were not in the MSID.
	
Criteria:	The 2008-09 FDOE School Transportation Monitoring District Self-Evaluation document states, in part:
“School Bus Evacuation Drills
Documentation was sufficient to confirm that all students who regularly ride a school bus and all persons qualified to transport students participated in school bus emergency evacuation drills on buses serving the school during the first six weeks of each semester, and all passengers on field and activity trips received instructions on the location and proper use of emergency exits prior to such trips in accordance with law and rule . . .
School Loading Zones for Buses
Criterion
Supervision was provided and school bus loading zones were designed and located to minimize hazards to students.  Vehicular traffic directional and warning signs were posted and traffic was monitored to ensure an orderly and safe flow . . .”
	
Cause:	STMS used the schools listing from district-maintained Web sites.  These sites often did not include the most current information or information on schools was intentionally left off websites.
	
Effect:	Inaccurate and incomplete reporting for monitoring of school busing safety issues.  All the schools which should be monitored are not included.
	
Recommendation:	STMS should instruct the districts to use the MSID as a basis for reconciling and determining a complete list of schools to be monitored.
	
Management
Response:	Beginning in 2006-07 STMS staff used the DOE school listings located at  the DOE Public Schools website as a basis for reviewing the evacuation drill and loading zone criteria, and supplemented this list with the district-provided listings cited in the Condition and Cause statements above.  As noted in the Finding 1 Condition statement, these lists include some schools that are not served by school bus, and thus are not required to be included in the STMS monitoring sample.  Conversely, some contracted centers that are not in the MSID or other school listings are served by school bus.  In Miami-Dade, under contracts with service providers, school buses serve children for whom evacuation drills must be performed at locations that are not “schools” listed on the MSID list.  Examples of two such locations are the University of Miami Mailman Center and the Easter Seals Center, both of which serve transported special needs students.  While the MSID comprises the most comprehensive starting point for determining the STMS sample gaining a 100% accurate listing of all schools and service locations served by school buses will require additional research by STMS staff working in conjunction with the districts.  All current and future monitoring reviews are being performed using this level of scrutiny to be sure that every school and service location subject to the referenced monitoring criteria is monitored.	

Finding 2 – Record keeping was not complete.

Condition:	
STMS files did not include the self-evaluation documentation for Pasco and Pinellas School Districts.  The only documentation found was for the STMS onsite review.
	
Criteria:	We were informed that, while unwritten, STMS policy is to maintain all self-evaluation documentation until a district’s next self-evaluation is completed.  Record retention policy should provide adequate documentation of work performed.
	
Cause:	Inadequate record keeping efforts contrary to an effective record retention policy.
	
Effect:	Inability to review a district’s previous documentation and substantiate that STMS did perform a review of the district.
	
Recommendation:	Ensure Self-Evaluation documentation is maintained until the next review is completed.
	
Management
Response:	Additional documentation of many of the self-evaluation records cited as missing for Pasco and Pinellas was subsequently provided to the auditor.  While copies of some district-submitted self-evaluation records for Pasco and Pinellas could not be located, the STMS monitors’ own review of the district self-evaluation was documented thoroughly; the STMS review provided redundancy and did provide verifiable evidence of district compliance in those areas where a portion of the two districts’ self-evaluation records were not located.  The STMS has implemented a tracking chart requiring physical review and documentation of the presence of all required records by a staff specialist and a supervisor to ensure that all self-evaluation documentation required from districts is maintained on the specified retention schedule.  Moreover, all district self-evaluation records are now scanned and maintained electronically, and original hard copies are filed in a locked cabinet, to ensure further redundancy and security.
	
Finding 3 – Special Needs requirements were not followed by the districts.

Condition:	For the past four years, the Special Needs section has been part of the self-evaluation although it was not included as part of the final report by STMS to each district.  Our review identified eight districts that did not forward any documentation as part of their self-evaluation.  Of those that did report, seven did not follow the sampling instructions which made the documentation of little use for properly evaluating Special Needs.  There are three parts to the Special Needs review which should be based on 10% of all Special Needs drivers.  Each successive part is supposed to be based on the previous sample.  The districts chose new random samples from the entire population for the two successive parts.  The lack of documentation and inadequate sampling efforts seem to indicate a general lack of understanding by the districts of the purpose of the Special Needs self-evaluation. 
	
Criteria:	The 2008-09 FDOE School Transportation Monitoring District Self-Evaluation document states, in part:
“Operator and Attendant Training Sample Size:
The minimum sample size must be ten percent of the bus operators and ten percent of attendants who transport students for which weighted funding is claimed, with no fewer than five operators and attendants . . .

IEP(Individual Education Plan) Compliance Sample Size:
NOTE: Students selected should be regular riders on buses/routes of operators/attendants that are in the operator/attendant sample from above.
Special Needs Bus Equipment and Use Sample Size:
Buses selected should be ones that are assigned to the operators/attendants that are in the operator/attendant sample from above.”
	
Cause:	This is a new area for districts; the majority of the rules are Federal. 
 	
Effect:	Little useful information about districts self-monitoring of Special Needs could be determined, thereby potentially placing students at risk.
	
Recommendation:	STMS should initiate training to inform and instruct districts on their duties in self-evaluation of Special Needs.
	
Management
Response:	The STMS special needs transportation specialist has clarified the written instructions and improved the content of the training that he has always provided to school districts regarding self-monitoring of transportation for Special Needs students.  The STMS reviews all self- evaluation records pursuant to this criterion and notifies districts that they must resubmit any records that are improperly completed.  Additionally, STMS maintains a log of submissions and their adequacy to ensure that all required documents are received and reviewed prior to the release of the final report.  It should be noted that in prior years and through the first half of 2008-09, the STMS staff also conducted on-site monitoring in this area to ensure redundancy and third-party oversight of district performance.	

Finding 4 – Some districts did not follow documentation directions or did not comply with rules.

Condition:	As noted in the first finding, over half of the sampled districts did not provide complete documentation on their schools.  We also noted the following:
•	87 schools in four districts with School Loading Zones for Buses deficiencies did not contain complete Corrective Action Plans (CAP) with an anticipated completion date.  Without an anticipated completion date, the CAP is not a completed plan.
•	168 schools in five districts submitted School Bus Evacuation Drill documentation showing the drills were not performed within the first six weeks of each semester.
•	Nine districts did not follow sampling instructions and performed more work than necessary.
•	We noted instances where two sets of Evacuation Drill worksheets for the same schools had been submitted which contradicted each other, and a number of the documents from both sets did not contain the required principal’s signature and date.	
Criteria:	2008-09 FDOE School Transportation Monitoring District Self-Evaluation and Section 6A-3.0171(2)(g)2d, Florida Administrative Code which states:
“Responsibilities of the school principal or other designated school staff:
. . .
To direct school bus emergency evacuation drills on each bus serving the school during the first six (6) weeks of each semester, and to maintain documentation for all students.”
	
Cause:	Inattention to detail and instructions.
	
Effect:	Rider safety may be diminished.
	
Recommendation:	STMS should:
•	Emphasize to districts the need to follow instructions and ensure Corrective Action Plans are complete.
•	Emphasize the need to perform Evacuation Drills at the beginning of the semesters.
•	Remind districts of limits on sampling for Operator Licensing and Qualifications, Bus Inspection Records, and Bus Safety Inspections.
•	Ensure documentation is signed and dated by the appropriate district employee.
	
Management
Response:	It should be noted that the statement that “over half of the sampled districts did not provide complete documentation on their schools,” while technically correct, overstates the degree of non-compliance by the districts relating to these criteria.  Since the sample size is every school in every district, being even a week late performing an evacuation drill at one school in a large district with over 300 schools puts that district in the status of not providing complete documentation.  The sample size is set at 100% because it is important that every required evacuation drill be performed on schedule at every school every time, but it may be more useful to state the percentage of overall compliance in each district, which is usually very high, than to provide the noted summary statement leading the reader to believe that half of all school districts are generally deficient.

Each year the STMS conducts training and reviews its self-evaluation work paper instructions to emphasize the need for districts to comply with the laws and rules being monitored and to improve compliance with requirements.  These requirements include completing Corrective Action Plans, performing Evacuation Drills on the required schedule, adhering to instructions for sampling limits, and adhering to instructions directing appropriate employees to sign and date related documents.  STMS staff and supervisors follow up on submitted Corrective Action Plans to be sure they are adequate to remedy any identified deficiencies and have implemented additional end-of-year review and follow-up communications to be sure all identified actions have been completed.	
OIG Comment:	We provided a listing of specific conditions found in the documentation reviewed related to the 20 school districts in our sample.  These conditions led us to the recommendations we offered.