At the center of “Saint Omer,” the impressive, if sometimes frustrating, debut feature by Alice Diop, is a remarkable performance by Guslagie Malanda. Although Ms. Malanda appeared in an episode of “The Romanoffs,” the much-maligned Amazon series that bears serious reconsideration, and assayed the title role in “My Friend Victoria” (2014), she is, by no stretch of the imagination, a seasoned actress.