Created by: Dpw506
Hello, could you please tell me whether the rotary Embeddings you wrote refer to Roformer?