LCL: COCOA: IME fully supported on MacOS
Issue
Lazarus does not support IME on MacOS, such as Chinese/Japanese/Korean.
(the terms LCL, WidgetSet, and Interface are prone to ambiguity and will not be used alone in the following)
Cause
the underlying cause is that in current Cocoa:
- LCL Edit Component (such as
SynEdit
/ATSynEdit
), corresponds to TCocoaCustomControl in Cocoa, which has not yet implemented the complete NSTextInputClient protocol -
Key Down Events
have been processed by LCL Edit Component as ordinary characters before being composed by NSInputContext. it's mainly caused by the current processing logic of TCocoaApplication.sendevent() and TLCLCommonCallback.KeyEvBefore()
Ideas
the idea is clear after the cause is clear:
- it's necessary to fully implement the NSTextInputClient protocol (the part related to Marked Text) in a suitable class
-
Key Events
related to IME need to be processed by NSInputContext first, and then the composed text is sent to LCL Component
Principles of the implementation
- the principle of minimum change: do not change the LCL, do not add any Message, and all changes are limited to Cocoa. since this is a large patch, it is necessary to reduce the review workload as much as possible.
- the principle of high cohesion and less coupling: each module performs its own duties, reduces coupling between modules, and minimizes dependencies
- the principle of elegance: avoid dealing with various IME Messages and various pointers in LCL Edit Component. it can be implemented straightforwardly by the Freepascal Interface.
Result
with the above principles and constraints, let's see how far we can go.
1. following the above principles, IME are fully supported in Cocoa
- various IME are fully supported, such as Chinese/Japanese/Korean, DeadKeys, Emoji & Symbols. no matter whether the IME has a popup window, whether popup halfway, or whether change the position of the popup window halfway.
- MultiCarets are supported in LCL Edit Component
- GroupUndo or not are both fully supported in LCL Edit Component
2. languages and IME tested
- Chinese: PinYin Simplified Chinese (MacOS buildin)
- Chinese: Baidu PinYin Simplified Chinese (Third Party)
- Chinese: Cangjie Traditional Chinese (MacOS buildin)
- Japanese: Kana (MacOS buildin)
- Japanese: Romaji (MacOS buildin)
- Korean: 2-Set (MacOS buildin)
- Dead Keys: ABC (MacOS buildin)
- Emoji & Symbols: Character Viewer (MacOS buildin)
3. two LCL Components have been implemented as MacOS IME samples
Description of the implementation
although the specific implementation involves modifications in many codes, the whole logic is straightforward.
it's in three layers: Cocoa, CocoaWS, LCL Component:
1. Cocoa Layer
- new FreePascal Interface added:
ICocoaIMEControl
(in unit CocoaPrivate)
As a bridge between Cocoa, CocoaWS and Component, decoupling the dependencies of each layer
// the LCL Component that need Cocoa IME support need to
// implement this simple interface
// class LazSynCocoaIMM in SynEdit Component for reference
// class ATSynEdit_Adapter_CocoaIME in ATSynEdit Component for reference
ICocoaIMEControl = interface
procedure IMESessionBegin;
procedure IMESessionEnd;
{
update IME Intermediate Text, Key function for IME:
1. some IME do not have a popup window and rely on the Editor
to display the Intermediate Text
2. it means completely cancel the IME session if Intermediate Text is empty
3. it's First Call of IMEUpdateIntermediateText or IMEInsertFinalText
if isFirstCall=True
4. eat some chars if eatAmount>0 (such as DeadKeys)
}
procedure IMEUpdateIntermediateText( var params: TCocoaIMEParameters );
{
insert IME Final Text, Key function for IME:
1. called only when inputting via IME, otherwise handled by UTF8KeyPress()
2. when the IME input is finished, either IMEUpdateIntermediateText(with empty text)
is called, or IMEInsertFinalText(with final text) is called,
NOT the both
3. it's First call of IMEUpdateIntermediateText or IMEInsertFinalText
if isFirstCall=True
4. eat some chars if eatAmount>0 (such as DeadKeys)
}
procedure IMEInsertFinalText( var params: TCocoaIMEParameters );
function IMEGetTextBound( var params: TCocoaIMEParameters ) : TRect;
end;
// IME Parameters for Cocoa Interface internal and LCL Full Control Edit
// intentionally keep the Record type, emphasizing that it is only a simple type,
// only used as parameters, don‘t put into logical functions
TCocoaIMEParameters = record
text: ShortString; // Marked Text
textCharLength: Integer; // length in code point
textByteLength: Integer; // length in bytes
textNSLength: Integer; // length in code unit (NSString)
selectedStart: Integer; // selected range start in code point
selectedLength: Integer; // selected range length in code point
eatAmount: Integer; // delete char out of Marked Text
isFirstCall: Boolean; // if first in the IME session
end;
- new FreePascal Class added:
TCocoaFullControlEdit
(in unit CocoaPrivate)
inherit from TCocoaCustomControl and fully implement NSTextInputClient Protocol (the part related to IME).
only depends on ICocoaIMEControl, no other dependencies.
// backend of LCL Full Control Edit Component (such as SynEdit/ATSynEdit)
// Key Class for Cocoa IME support
// 1. obtain IME capability from Cocoa by implementing NSTextInputClientProtocol
// 2. synchronize IME data with LCL via ICocoaIMEControl
TCocoaFullControlEdit = objcclass(TCocoaCustomControl)
private
_currentParams: TCocoaIMEParameters;
_currentMarkedText: NSString;
public
imeHandler: ICocoaIMEControl;
public
{
for IME Key Down:
Key step for IME (such as Chinese/Japanese/Korean and DeadKeys)
1. forward key event to NSInputContext
2. NSInputContext will call TCocoaFullControlEdit(NSTextControlClient)
and then call LCL via imeHandler
}
procedure keyDown(theEvent: NSEvent); override;
{
for IME Close:
1. mouseDown() will not be called when click in the IME Popup Window,
so it must be clicking outside the IME Popup Windows,
which should end the IME input
2. Cocoa had called setMarkedText_selectedRange_replacementRange()
or insertText_replacementRange() first, then mouseDown() here
3. NSInputContext.handleEvent() just close IME window here
4. LCL actually handle mouse event
}
procedure mouseDown(event: NSEvent); override;
procedure mouseUp(event: NSEvent); override;
function resignFirstResponder: ObjCBOOL; override;
{
send Marked/Intermediate Text to LCL Edit Control which has IME Handler
Key step for IME (such as Chinese/Japanese/Korean and DeadKeys)
}
procedure setMarkedText_selectedRange_replacementRange (aString: id; newRange: NSRange; replacementRange: NSRange); override;
{
send final Text to LCL Edit Control which has IME Handler
Key step for IME (such as Chinese/Japanese/Korean and DeadKeys)
1. if in IME input state, handle text via imeHandler.IMEInsertFinalText()
2. otherwise via lclGetCallback.InputClientInsertText,
mainly for maximum backward compatibility with TCocoaCustomControl
}
procedure insertText_replacementRange (aString: id; replacementRange: NSRange); override;
procedure unmarkText; override;
function markedRange: NSRange; override;
function selectedRange: NSRange; override;
function hasMarkedText: LCLObjCBoolean; override;
// cursor tracking
function firstRectForCharacterRange_actualRange ({%H-}aRange: NSRange; {%H-}actualRange: NSRangePointer): NSRect; override;
end;
-
TCocoaApplication
improved (in unit CocoaInt)
improved key event handling mechanism by NSTextInputClientProtocol.hasMarkedText() in IME input state.
adjust the priority of Key Event Handlers, Cocoa firstif hasMarkedText=true
, otherwise keep LCL first unchanged from the original.
procedure TCocoaApplication.sendEvent(theEvent: NSEvent);
......
if Assigned(win) then
begin
responder := win.firstResponder;
cb := responder.lclGetCallback;
if Assigned(cb) and (theEvent.type_=NSKeyDown) then
begin
// set CocoaOnlyState when NSKeyDown only,
// keep last CocoaOnlyState when NSKeyUp
if responder.conformsToProtocol(objcprotocol(NSTextInputClientProtocol)) then
cb.CocoaOnlyState := NSTextInputClientProtocol(responder).hasMarkedText
else
cb.CocoaOnlyState := false;
end;
......
if cb.IsCocoaOnlyState then
begin
inherited sendEvent(theEvent);
end
else
begin
cb.KeyEvBefore(theEvent, allowcocoa);
if allowcocoa then
inherited sendEvent(theEvent);
cb.KeyEvAfter;
end;
......
2. CocoaWS Layer
- new FreePascal Interface added:
TLCLFullControlEditCallBack
(in unit CocoaWsCommon)
inherit from TLCLCommonCallBack. by adjusting KeyEvPrepare() to avoidKey Events
being processed by LCL Edit Component as ordinary characters before being composed by NSInputContext.
// CallBack for LCL Full Control Edit (such as SynEdit/ATSynEdit)
TLCLFullControlEditCallBack = class(TLCLCommonCallBack)
protected
{
Key Step for IME (such as Chinese/Japanese/Korean and DeadKeys)
1. set _sendChar:=false to avoid KeyDown Event being eaten
in IntfUTF8KeyPress() or CN_CHAR message.
2. KeyDown Event will be handled in TCocoaFullControlEdit.keyDown(),
and NSInputContext.sendEvent() will be called in it,
and function in NSTextInputClient will be called.
}
procedure KeyEvPrepare(Event: NSEvent); override;
end;
-
TCocoaWSCustomControl
improved (in unit CocoaWsCommon)
create different Cocoa Handle according to whether the LCL Component supports ICocoaIMEControl.
create TCocoaFullControlEdit as Handle if AWinControl supports ICocoaIMEControl, otherwise keep creating TCocoaCustomControl unchanged from the original.
// get IMEHandler by LM_IM_COMPOSITION message
function getControlIMEHandler(const control: TWinControl): ICocoaIMEControl;
var
handle : PtrInt;
begin
handle := SendSimpleMessage(control, LM_IM_COMPOSITION);
Result := TObject(handle) as ICocoaIMEControl
end;
class function TCocoaWSCustomControl.CreateHandle(const AWinControl: TWinControl;
const AParams: TCreateParams): TLCLIntfHandle;
var
ctrl : TCocoaCustomControl;
......
imeHandler : ICocoaIMEControl;
begin
imeHandler := getControlIMEHandler(AWinControl);
if Assigned(imeHandler) then
begin
// AWinControl implements ICocoaIMEControl
// AWinControl is a Full Control Edit (such as SynEdit/ATSynEdit)
ctrl := TCocoaFullControlEdit.alloc.lclInitWithCreateParams(AParams);
lcl := TLCLFullControlEditCallback.Create(ctrl, AWinControl);
TCocoaFullControlEdit(ctrl).imeHandler := imeHandler;
ctrl.unmarkText;
end
else
begin
// AWinControl not implements ICocoaIMEControl
// AWinControl is a normal Custom Control
ctrl := TCocoaCustomControl.alloc.lclInitWithCreateParams(AParams);
lcl := TLCLCommonCallback.Create(ctrl, AWinControl);
end;
......
end;
3. Component Layer
- SynEdit @martin_frb
the Cocoa IME quality is the same as the Windows version, and additionally supports MultiCarets.
new unit LazSynCocoaIMM added, which only depends on ICocoaIMEControl, with minimal intrusion.
it's implemented straightforwardly by the Freepascal Interface, without annoying details of various Messages.
unit LazSynCocoaIMM
{
SynEdit MacOS IME Handler:
1. various IME are fully supported, such as Chinese/Japanese/Korean and DeadKeys
2. MultiCarets supported
3. GroupUndo or not are both fully supported
}
LazSynImeCocoa = class( LazSynIme, ICocoaIMEControl )
private
_undoList: TSynEditUndoList;
_IntermediateTextBeginPos: TPoint;
public
procedure IMESessionBegin;
procedure IMESessionEnd;
procedure IMEUpdateIntermediateText( var params: TCocoaIMEParameters );
procedure IMEInsertFinalText( var params: TCocoaIMEParameters );
function IMEGetTextBound( var params: TCocoaIMEParameters ) : TRect;
private
procedure InsertTextAtCaret_CompatibleWithMultiCarets( var params: TCocoaIMEParameters ) ;
procedure SelectText_CompatibleWithMultiCarets( var params: TCocoaIMEParameters );
function calcBound( var params: TCocoaIMEParameters ) : TRect;
function PosToPixels( const pos: TPoint ) : TPoint;
public
constructor Create(AOwner: TSynEditBase);
destructor Destroy; override;
end;
SynEdit only needs to respond to the LM_IM_COMPOSITION message and return the LazSynImeCocoa instance
unit SynEdit
TCustomSynEdit = class(TSynEditBase)
......
private
procedure COCOA_IMComposition(var Message: TMessage); message LM_IM_COMPOSITION;
......
end;
constructor TCustomSynEdit.Create(AOwner: TComponent);
begin
......
FImeHandler := LazSynImeCocoa.Create(Self);
......
end;
procedure TCustomSynEdit.Cocoa_IMComposition(var Message: TMessage);
begin
Message.Result := PtrInt(FImeHandler);
end;
- ATSynEdit @Alexey-T1
the IME quality close to mainstream native text editors (such as TextMate), better on Undo/Redo.
new unit ATSynEdit_Adapter_CocoaIME added, which only depends on ICocoaIMEControl, without annoying details of various Messages.
unit ATSynEdit_Adapter_CocoaIME
{
ATSynEdit MacOS IME Adapter:
1. various IME are fully supported, such as Chinese/Japanese/Korean and DeadKeys
2. MultiCarets and MultiSelections fully supported
3. GroupUndo or not are both fully supported
}
TATAdapterCocoaIME = class( TATAdapterIME, ICocoaIMEControl )
private
_editor: TATSynEdit;
_IntermediateTextBeginPos: TPoint;
public
procedure IMESessionBegin;
procedure IMESessionEnd;
procedure IMEUpdateIntermediateText( var params: TCocoaIMEParameters );
procedure IMEInsertFinalText( var params: TCocoaIMEParameters );
function IMEGetTextBound( var params: TCocoaIMEParameters ) : TRect;
private
procedure selectIntermediateText( var params: TCocoaIMEParameters );
function calcBound( var params: TCocoaIMEParameters ) : TRect;
function getSuitableCaret() : TATCaretItem;
public
constructor Create( const editor: TATSynEdit );
destructor Destroy; override;
end;
Patches
for clarity, the relevant code is split into 5 patches:
- FIX: COCOA: improved key event handling mechanism by NSTextInputClientProtocol.hasMarkedText() in IME input state 1. IME patch for Cocoa
A PR !116 (merged) has been submitted before, but has not been merged. - LCL: COCOA: ADD IME fully support (1/2): implements NSTextInputClient in TCocoaFullControlEdit 2. IME patch for Cocoa
- LCL: COCOA: ADD IME fully support (2/2): create TCocoaFullControlEdit in TCocoaWSCustomControl.CreateHandle() 3. IME patch for Cocoa
- LCL/SynEdit: ADD: MacOS IME fully supported 4. IME patch for SynEdit
- LCL/ATSynEdit: ADD: Cocoa: MacOS IME fully supported (in the repository of ATSynEdit) 5. IME patch for TASynEdit
Examples
-
testSynEditIME:testSynEditIME.zip @martin_frb
This example simply provides two TSynEdits in one form. -
Lazarus IDE:@martin_frb
the Lazarus IDE itself uses the SynEdit component, just recompile with the new SynEdit.
fully tested for two weeks. -
ATSynEdit/app/demo_editor:@Alexey-T1
the official demo for ATSynEdit, various settings of ATSynEdit can be tested.
fully testd for two weeks. -
CudaText: @Alexey-T1
a great plain text editor, CudaText uses ATSynEdit component, just recompile with the new ATSynEdit.
References
- Cocoa Internals
- Cocoa Internals/Input
- LCL Key Handling
- SynEdit
- ATSynEdit
- thanks @boramis and @skalogryz very much for the detailed documents and the source code related to Cocoa, and many discussions and ideas about IME.
Edited by rich2014