Skip to content
This repository has been archived by the owner on Jun 12, 2018. It is now read-only.

Application Multiplexer #844

Merged
merged 56 commits into from
Feb 25, 2014
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
56 commits
Select commit Hold shift + click to select a range
777e4b3
Create initial module layout, based on the keyword router
Feb 4, 2014
6a5379f
fix requirements.pip and add secret key
Feb 7, 2014
8c97624
Merge branch 'develop' of github.com:praekelt/vumi-go into feature/is…
Feb 7, 2014
eadd57d
Tweak logging in settings.py
Feb 7, 2014
e920803
Merge branch 'develop' of github.com:praekelt/vumi-go into feature/is…
Feb 11, 2014
8ada227
Implemented django forms for the router detail view
Feb 11, 2014
4ba07ed
Improved validation of user input
Feb 14, 2014
4430cf8
add tests for router config validation
Feb 14, 2014
f4784c6
hmm, hacked go.testsettings for some logging issue
Feb 14, 2014
cce9f7f
Add non_field_errors to router edit template
Feb 14, 2014
a9b1875
Rename multiplexer directory for more consistency
Feb 17, 2014
cdb71d1
Change various config-related field names
Feb 17, 2014
dfa3e67
Move all form-related stuff to a new forms.py file
Feb 17, 2014
c2ab39d
remove local dev hacks from the various configs
Feb 17, 2014
26ca348
Update tests as per changes in the view definition
Feb 17, 2014
370dd5d
Get tests working again after modifications to the view
Feb 17, 2014
e4cf441
Only include non_field_errors if there actually errors
Feb 18, 2014
5000b2d
Merge branch 'develop' of github.com:praekelt/vumi-go into feature/is…
Feb 18, 2014
a68f96c
remove some obsolete Nose stuff which crept in via a bad merge
Feb 18, 2014
3f2656a
Flesh out the router worker to implement basic routing
Feb 19, 2014
e289419
Add tests for various helpers
Feb 19, 2014
1eb8a7b
Remove empty setup/teardown routines that do nothing
Feb 19, 2014
581c142
simplify verification of choosen endpoint
Feb 19, 2014
7dfe830
Fix typo in the router edit template
Feb 19, 2014
b5cee25
Revert changes to the magic comments in vumi_app.py
Feb 19, 2014
e000b41
Remove irrelevant whitespace fix from go.settings
Feb 19, 2014
0c52ebf
Some cleanups and refactoring in the multiplexer worker
Feb 20, 2014
eaa947d
Add a reference to a Vumi ticket in common.py
Feb 20, 2014
9db74c7
Cleanup existing tests and sort out issues from review
Feb 20, 2014
52cceec
Cleanup/iterate on router configuration as per the the review
Feb 20, 2014
0be9e4c
Add test for the START state
Feb 20, 2014
db58f15
Add test helper and a bunch of tests
Feb 20, 2014
97c9bb6
More cleanups, and fixes for various bugs
Feb 20, 2014
6577351
More robust endpoint selection
Feb 21, 2014
216f447
Rewrite some of the tests as per Jeremy's suggestions
Feb 21, 2014
eafccbe
Finish rewriting tests for the app multiplexor worker
Feb 21, 2014
b2be461
Change app multiplexer module namespace to 'go.routers.app_multiplexer'
Feb 24, 2014
b894b25
Simplify state transition handling
Feb 24, 2014
0780889
Use a global session manager based on static config
Feb 24, 2014
aeceb82
Remove unnecessary newlines from multiplexer tests
Feb 24, 2014
8331801
Remove keyword scanning feature from multiplexer
Feb 24, 2014
7a81223
handle session 'close' messages in both directions
Feb 24, 2014
dd6d269
Add tests for handling of 'close' messages
Feb 24, 2014
280404e
Last batch of tests for handling 'close' messages
Feb 24, 2014
22e2192
Minor test cleanups for multiplexer
Feb 24, 2014
29c7306
Remove unnecessary ABORT state
Feb 24, 2014
4ce79e6
change multiplexer namespace to 'app_multiplexer'
Feb 24, 2014
b5f9dbf
Revert to using a dynamic session manager
Feb 25, 2014
7fe7f3e
remove unused smart_truncate function
Feb 25, 2014
45f53b4
Reduce session expiration to 5min for multiplexor sessions
Feb 25, 2014
e3b47bf
Various minor cleanups as per review comments
Feb 25, 2014
3727f56
Just forward a 'close' message without trying to modify it
Feb 25, 2014
a2d2a00
Cleanup up some multiplexer tests, add test docs, etc
Feb 25, 2014
f19ab98
Use dict literals instead of dict constructors in the mutiplexer
Feb 25, 2014
aa55b10
De-inline the handlers for the 'None' next_state
Feb 25, 2014
098b6cc
Make the router key a component of session keys
Feb 25, 2014
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions go/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,10 @@ def get_router_definition(router_type, router=None):
'namespace': 'keyword',
'display_name': 'Keyword',
},
'go.routers.app_multiplexer': {
'namespace': 'application_multiplexer',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's probably best if the namespace matches the module name, i.e. app_multiplexer. We don't do that in a few places but those are all places which we want to get rid of eventually.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The namespace value needs to change to match the module name.

'display_name': 'Application Multiplexer',
},
}

_VUMI_OBSOLETE_ROUTERS = [
Expand Down
Empty file.
8 changes: 8 additions & 0 deletions go/routers/app_multiplexer/definition.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
from go.vumitools.router.definition import RouterDefinitionBase


class RouterDefinition(RouterDefinitionBase):
router_type = 'application_multiplexer'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will need to be updated when we update the namespace name.


def configured_outbound_endpoints(self, config):
return list(set(config.get('keyword_endpoint_mapping', {}).values()))
Empty file.
24 changes: 24 additions & 0 deletions go/routers/app_multiplexer/tests/test_views.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
from go.base.tests.helpers import GoDjangoTestCase
from go.routers.tests.view_helpers import RouterViewsHelper


class ApplicationMultiplexerViewTests(GoDjangoTestCase):

def setUp(self):
self.router_helper = self.add_helper(
RouterViewsHelper(u'application_multiplexer')
)
self.user_helper = self.router_helper.vumi_helper.get_or_create_user()
self.client = self.router_helper.get_client()

def test_new_router(self):
router_store = self.user_helper.user_api.router_store
self.assertEqual([], router_store.list_routers())

response = self.client.post(self.router_helper.get_new_view_url(), {
'name': u"myrouter",
'router_type': u'application_multiplexer',
})
[router_key] = router_store.list_routers()
rtr_helper = self.router_helper.get_router_helper_by_key(router_key)
self.assertRedirects(response, rtr_helper.get_view_url('edit'))
71 changes: 71 additions & 0 deletions go/routers/app_multiplexer/tests/test_vumi_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
from twisted.internet.defer import inlineCallbacks

from vumi.tests.helpers import VumiTestCase

from go.routers.app_multiplexer.vumi_app import ApplicationMultiplexer
from go.routers.tests.helpers import RouterWorkerHelper


class TestApplicationMultiplexerRouter(VumiTestCase):

router_class = ApplicationMultiplexer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This attribute is unnecessary and should be removed.


@inlineCallbacks
def setUp(self):
self.router_helper = self.add_helper(
RouterWorkerHelper(ApplicationMultiplexer)
)
self.router_worker = yield self.router_helper.get_router_worker({})

@inlineCallbacks
def assert_routed_inbound(self, content, router, expected_endpoint):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing calls this. Can it be removed?

msg = yield self.router_helper.ri.make_dispatch_inbound(
content, router=router)
emsg = msg.copy()
emsg.set_routing_endpoint(expected_endpoint)
rmsg = self.router_helper.ro.get_dispatched_inbound()[-1]
self.assertEqual(emsg, rmsg)

@inlineCallbacks
def test_start(self):
router = yield self.router_helper.create_router()
self.assertTrue(router.stopped())
self.assertFalse(router.running())

yield self.router_helper.start_router(router)
router = yield self.router_helper.get_router(router.key)
self.assertFalse(router.stopped())
self.assertTrue(router.running())

@inlineCallbacks
def test_stop(self):
router = yield self.router_helper.create_router(started=True)
self.assertFalse(router.stopped())
self.assertTrue(router.running())

yield self.router_helper.stop_router(router)
router = yield self.router_helper.get_router(router.key)
self.assertTrue(router.stopped())
self.assertFalse(router.running())

@inlineCallbacks
def test_no_messages_processed_while_stopped(self):
router = yield self.router_helper.create_router()

yield self.router_helper.ri.make_dispatch_inbound("foo", router=router)
self.assertEqual([], self.router_helper.ro.get_dispatched_inbound())

yield self.router_helper.ri.make_dispatch_ack(router=router)
self.assertEqual([], self.router_helper.ro.get_dispatched_events())

yield self.router_helper.ro.make_dispatch_outbound(
"foo", router=router)
self.assertEqual([], self.router_helper.ri.get_dispatched_outbound())
[nack] = self.router_helper.ro.get_dispatched_events()
self.assertEqual(nack['event_type'], 'nack')

@inlineCallbacks
def test_inbound_no_config(self):
router = yield self.router_helper.create_router(started=True)
yield self.assert_routed_inbound("foo bar", router, 'default')
yield self.assert_routed_inbound("baz quux", router, 'default')
42 changes: 42 additions & 0 deletions go/routers/app_multiplexer/view_definition.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
from django import forms

from go.router.view_definition import RouterViewDefinitionBase, EditRouterView


class ApplicationMultiplexerForm(forms.Form):
keyword = forms.CharField()
target_endpoint = forms.CharField()


class BaseApplicationMultiplexerFormSet(forms.formsets.BaseFormSet):
@staticmethod
def initial_from_config(data):
return [{'keyword': k, 'target_endpoint': v}
for k, v in sorted(data.items())]

def to_config(self):
keyword_endpoint_mapping = {}
for form in self:
if (not form.is_valid()) or form.cleaned_data['DELETE']:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Elsewhere we use if not form.cleaned_data or form in self.deleted_forms here which might be a better check (the form will already have been validated by the time to_config is called and deleted_forms will have been populated).

continue
keyword = form.cleaned_data['keyword']
target_endpoint = form.cleaned_data['target_endpoint']
keyword_endpoint_mapping[keyword] = target_endpoint
return keyword_endpoint_mapping


ApplicationMultiplexerFormSet = forms.formsets.formset_factory(
ApplicationMultiplexerForm,
can_delete=True,
extra=1,
formset=BaseApplicationMultiplexerFormSet)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest putting these forms in a forms.py module and keeping this module for the views (just to keep things tidy and a bit more like Django coders expect). Also allows gives a nice place (i.e. tests/test_forms.py) to put direct tests of the form methods.



class EditApplicationMultiplexerView(EditRouterView):
edit_forms = (
('keyword_endpoint_mapping', ApplicationMultiplexerFormSet),
)


class RouterViewDefinition(RouterViewDefinitionBase):
edit_view = EditApplicationMultiplexerView
37 changes: 37 additions & 0 deletions go/routers/app_multiplexer/vumi_app.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
# -*- test-case-name: go.routers.keyword.tests.test_vumi_app -*-
# -*- coding: utf-8 -*-

from vumi import log
from vumi.config import ConfigDict

from go.vumitools.app_worker import GoRouterWorker


class ApplicationMultiplexerConfig(GoRouterWorker.CONFIG_CLASS):
keyword_endpoint_mapping = ConfigDict(
"Mapping from case-insensitive keyword regex to endpoint name.",
default={})


class ApplicationMultiplexer(GoRouterWorker):
"""
Router that splits inbound messages based on keywords.
"""
CONFIG_CLASS = ApplicationMultiplexerConfig

worker_name = 'application_multiplexer'

def lookup_target(self, config, msg):
first_word = ((msg['content'] or '').strip().split() + [''])[0]
for keyword, target in config.keyword_endpoint_mapping.iteritems():
if keyword.lower() == first_word.lower():
return target
return 'default'

def handle_inbound(self, config, msg, conn_name):
log.debug("Handling inbound: %s" % (msg,))
return self.publish_inbound(msg, self.lookup_target(config, msg))

def handle_outbound(self, config, msg, conn_name):
log.debug("Handling outbound: %s" % (msg,))
return self.publish_outbound(msg, 'default')