Section 1355.33 - Procedures for the review(a) The full child and family services reviews will:(1) Consist of a two-phase process that includes a statewide assessment and an on-site review; and(2) Be conducted by a team of Federal, and State or Tribal reviewers that includes: (i) Staff of the child and family services agency, including the offices that represent the service areas that are the focus of any particular review;(ii) Representatives selected by the title IV-E agency, in collaboration with the ACF Regional Office, from those with whom the title IV-E agency was required to consult in developing its CFSP, as described and required in 45 CFR 1357.15(l) ;(iii) Federal staff of HHS; and(iv) Other individuals, as deemed appropriate and agreed upon by the title IV-E agency and ACF.(b)Statewide or Tribal Assessment. The first phase of the full review will be a statewide assessment, or for a Tribal title IV-E agency a service area assessment, conducted by the title IV-E agency's internal and external members of the review team. The assessment must: (1) Address each systemic factor under review including the statewide/Tribal information system; case review system; quality assurance system; staff training; service array; agency responsiveness to the community; and foster and adoptive parent licensing, recruitment and retention;(2) Assess the outcome areas of safety, permanence, and well-being of children and families served by the title IV-E agency using data from AFCARS and NCANDS. For the initial review, ACF may approve another data source to substitute for AFCARS, and in all reviews, ACF may approve another data source to substitute for NCANDS. The title IV-E agency must also analyze and explain its performance in meeting the national standards for the statewide/Tribal service area data indicators;(3) Assess the characteristics of the title IV-E agency that have the most significant impact on the agency's capacity to deliver services to children and families that will lead to improved outcomes;(4) Assess the strengths and areas of the title IV-E agency's child and family services programs that require further examination through an on-site review;(5) Include a listing of all the persons external to the title IV-E agency who participated in the preparation of the assessment pursuant to § 1355.33(a)(2)(ii) and (iv) ; and(6) Be completed and submitted to ACF within 4 months of the date that ACF transmits the information for the assessment to the title IV-E agency.(c)On-site review. The second phase of the full review will be an on-site review. (1) The on-site review will cover the title IV-E agency's programs under titles IV-B and IV-E of the Act, including in-home services and foster care. It will be jointly planned by the title IV-E agency and ACF, and guided by information in the completed assessment that identifies areas in need of improvement or further review.(2) The on-site review may be concentrated in several specific political subdivisions or jurisdictions of the title IV-E agency, as agreed upon by the ACF and the title IV-E agency; however, for a State title IV-E agency, a State's largest metropolitan subdivision must be one of the locations selected.(3) ACF has final approval of the selection of specific areas of the title IV-E agency's child and family services continuum described in paragraph (c)(1) of this section and selection of the political subdivisions or jurisdiction referenced in paragraph (c)(2) of this section.(4) Sources of information collected during the on-site review to determine substantial conformity must include, but are not limited to:(i) Case records on children and families served by the agency;(ii) Interviews with children and families whose case records have been reviewed and who are, or have been, recipients of services of the agency;(iii) Interviews with caseworkers, foster parents, and service providers for the cases selected for the on-site review; and(iv) Interviews with key stakeholders, both internal and external to the agency, which, at a minimum, must include those individuals who participated in the development of the State's or Tribal title IV-E agency's CFSP required at 45 CFR 1357.15(1) , courts, administrative review bodies, children's guardians ad litem and other individuals or bodies assigned responsibility for representing the best interests of the child.(5) The sample will range from 30-50 cases. Foster care cases must be drawn randomly from AFCARS, or, for the initial review, from another source approved by ACF and include children who entered foster care during the year under review. In-home cases must be drawn randomly from NCANDS or from another source approved by ACF. To ensure that all program areas are adequately represented, the sample size may be increased.(6) The sample of 30-50 cases reviewed on-site will be selected from a randomly drawn oversample of no more than 150 foster care and 150 in-home services cases. The oversample must be statistically significant at a 90 percent compliance rate (95 percent in subsequent reviews), with a tolerable sampling error of 5 percent and a confidence coefficient of 95 percent. The additional cases in the oversample not selected for the on-site review will form the sample of cases to be reviewed, if needed, in order to resolve discrepancies between the statewide/Tribal assessment and the on-site review in accordance with paragraph (d)(2) of this section.(d)Resolution of discrepancies between the assessment and the findings of the on-site portion of the review. Discrepancies between the statewide or Tribal assessment and the findings of the on-site portion of the review will be resolved by either of the following means, at the title IV-E agency's option: (1) The submission of additional information by the title IV-E agency; or(2) ACF and the title IV-E agency will review additional cases using only those indicators in which the discrepancy occurred. ACF and the title IV-E agency will determine jointly the number of additional cases to be reviewed, not to exceed 150 foster care cases or 150 in-home services cases to be selected as specified in paragraph (c)(6) of this section.(e)Partial review. A partial child and family services review, when required, will be planned and conducted jointly by ACF and the title IV-E agency based on the nature of the concern. A partial review does not substitute for the full reviews as required under § 1355.32(b) .(f)Notification. Within 30 calendar days following either a partial child and family services review, full child and family services review, or the resolution of a discrepancy between the assessment and the findings of the on-site portion of the review, ACF will notify the title IV-E agency in writing of whether the title IV-E agency is, or is not, operating in substantial conformity.65 FR 4077, Jan. 25, 2000, as amended at 66 FR 58675, Nov. 23, 2001; 77 FR 927, Jan. 6, 2012