estimateTokenCount function

int estimateTokenCount(
  1. CodeMap codemap
)

Estimates the token count for a CodeMap object

This is useful for LLM context window planning

Implementation

int estimateTokenCount(CodeMap codemap) {
  final jsonString = jsonEncode(codemap.toJson());
  final tokens = jsonString
      .split(RegExp(r'[\s{}[\],:"]+'))
      .where((token) => token.isNotEmpty)
      .length;
  return tokens;
}