Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rework indexed grammars #15

Open
wants to merge 12 commits into
base: rework_cfg
Choose a base branch
from
Open

Rework indexed grammars #15

wants to merge 12 commits into from

Conversation

bygu4
Copy link
Collaborator

@bygu4 bygu4 commented Dec 5, 2024

Rework the remaining modules: fst and indexed_grammar. Use existing object representations for better typing, remove import cycles by refactoring the intersection methods. Add pyright type checker to requirements, update CI config by adding type checking jobs. Also split CI into different workflows to make the configuration clearer.

Changes by module:

indexed_grammar:

  • Rework rules by using CFG objects.
  • Refactor intersection method implementation by moving it to IndexedGrammar from FST.
  • Define intersection for FST explicitly.
  • Add utils file to contain nontrivial indexed grammar utility.

fst:

  • Rework FST by using finite automata object representations.
  • Rework the constructor of FST to make it similar to the other classes.
  • Add TransitionFunction class.
  • Add methods for adding and removing transitions.
  • Add methods for transition iteration and transition function call.
  • Add methods for copying of FST.
  • Move FST state renaming to utils file.

Copy link

github-actions bot commented Dec 6, 2024

Coverage

Coverage Report
FileStmtsMissCoverMissing
pyformlang
   __init__.py90100% 
pyformlang/cfg
   __init__.py30100% 
   cfg.py46622 99%
   cfg_variable_converter.py6544 94%
   cyk_table.py790100% 
   formal_grammar.py6911 99%
   llone_parser.py16333 98%
   parse_tree.py6511 98%
   recursive_decent_parser.py6122 97%
   set_queue.py150100% 
   utils.py250100% 
pyformlang/cfg/tests
   __init__.py00100% 
   test_cfg.py62622 99%
   test_llone_parser.py11711 99%
   test_production.py210100% 
   test_recursive_decent_parser.py2511 96%
   test_terminal.py330100% 
   test_variable.py180100% 
pyformlang/fcfg
   __init__.py40100% 
   fcfg.py13111 99%
   feature_production.py250100% 
   feature_structure.py19133 98%
   state.py360100% 
pyformlang/fcfg/tests
   __init__.py00100% 
   test_fcfg.py1690100% 
   test_feature_structure.py1590100% 
pyformlang/finite_automaton
   __init__.py70100% 
   deterministic_finite_automaton.py18333 98%
   deterministic_transition_function.py2411 96%
   doubly_linked_list.py350100% 
   doubly_linked_node.py100100% 
   epsilon_nfa.py21011 99%
   finite_automaton.py23111 99%
   hopcroft_processing_list.py240100% 
   nondeterministic_finite_automaton.py400100% 
   nondeterministic_transition_function.py480100% 
   partition.py400100% 
   transition_function.py320100% 
   utils.py300100% 
pyformlang/finite_automaton/tests
   __init__.py00100% 
   test_deterministic_finite_automaton.py2960100% 
   test_deterministic_transition_function.py8955 94%
   test_epsilon.py130100% 
   test_epsilon_nfa.py5950100% 
   test_nondeterministic_finite_automaton.py1600100% 
   test_nondeterministic_transition_function.py790100% 
   test_state.py280100% 
   test_symbol.py290100% 
pyformlang/fst
   __init__.py20100% 
   fst.py18611 99%
   transition_function.py3833 92%
   utils.py250100% 
pyformlang/fst/tests
   __init__.py00100% 
   test_fst.py1910100% 
pyformlang/indexed_grammar
   __init__.py90100% 
   consumption_rule.py3522 94%
   duplication_rule.py320100% 
   end_rule.py3311 97%
   indexed_grammar.py27111 99%
   production_rule.py3511 97%
   reduced_rule.py290100% 
   rule_ordering.py730100% 
   rules.py770100% 
   utils.py410100% 
pyformlang/indexed_grammar/tests
   __init__.py00100% 
   test_indexed_grammar.py2490100% 
   test_rules.py350100% 
pyformlang/objects
   __init__.py50100% 
   base_epsilon.py1511 93%
   base_terminal.py70100% 
   formal_object.py240100% 
pyformlang/objects/cfg_objects
   __init__.py60100% 
   cfg_object.py50100% 
   epsilon.py30100% 
   production.py4111 98%
   terminal.py100100% 
   utils.py140100% 
   variable.py130100% 
pyformlang/objects/finite_automaton_objects
   __init__.py50100% 
   epsilon.py30100% 
   finite_automaton_object.py50100% 
   state.py70100% 
   symbol.py50100% 
   utils.py140100% 
pyformlang/objects/pda_objects
   __init__.py60100% 
   epsilon.py30100% 
   pda_object.py50100% 
   stack_symbol.py70100% 
   state.py70100% 
   symbol.py50100% 
   utils.py2111 95%
pyformlang/objects/regex_objects
   __init__.py20100% 
   regex_objects.py630100% 
   utils.py220100% 
pyformlang/pda
   __init__.py40100% 
   pda.py31722 99%
   transition_function.py3933 92%
   utils.py5322 96%
pyformlang/pda/tests
   __init__.py00100% 
   test_pda.py3020100% 
pyformlang/regular_expression
   __init__.py30100% 
   python_regex.py26966 98%
   regex.py2811414 95%
   regex_reader.py15944 97%
pyformlang/regular_expression/tests
   __init__.py00100% 
   test_python_regex.py2780100% 
   test_regex.py4110100% 
pyformlang/rsa
   __init__.py30100% 
   box.py512525 51%
   recursive_automaton.py7299 88%
pyformlang/rsa/tests
   __init__.py00100% 
   test_rsa.py370100% 
pyformlang/tests
   __init__.py00100% 
TOTAL874110999% 

Tests Skipped Failures Errors Time
310 0 💤 0 ❌ 0 🔥 8.292s ⏱️

@bygu4 bygu4 requested a review from gsvgit December 6, 2024 13:47
Copy link
Member

@gsvgit gsvgit left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All docstrings should be updated too.

@@ -41,7 +41,7 @@ def test_production_rules(self):
""" Tests the production rules """
produ = ProductionRule("S", "C", "end")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not production?

from typing import Callable, List, Set, Iterable, Any


def exists(list_elements: List[Any],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it any(map(predicate, iterable)) ? As I know, map returns enumerator, not list, so this version should be more or less optimal.


@property
def right_terms(self) -> Tuple[Any, Any]:
def right_terms(self) -> List[CFGObject]:
"""Gives the non-terminals on the right of the rule

Returns
---------
right_terms : iterable of any
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

List of CFGObjects ?

is_duplication : bool
Whether the rule is a duplication rule or not
---------
left_term : any
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it Any actually?


@property
def non_terminals(self) -> Iterable[Any]:
def non_terminals(self) -> Set[Variable]:
"""Gets the non-terminals used in the rule

non_terminals : iterable of any
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Set?

def left_term(self) -> Variable:
"""Gets the symbol on the left of the rule

left : any
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any?


@property
def states(self):
def states(self) -> Set[State]:
""" Get the states of the FST

Returns
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

of State

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants