Commit Graph

1147 Commits

Author SHA1 Message Date
Luke Faraone 895b7dc9b1 Accept emails on stdin in email-mirror and queue for later processing
(imported from commit 7efccc294f87d3661e50f10962a80071b91e3879)
2014-01-15 16:07:18 -05:00
Luke Faraone fbb2b6945b Queue worker for email mirror processing.
(imported from commit b60049c84622ea457db38c5a5b185bcd108bc59f)
2014-01-15 16:07:18 -05:00
Luke Faraone 0b5036e0fb Let message recipient in process_message be passed as an argument.
(imported from commit cfcfb9edf748a35d279235ee11a05b1a00c934b5)
2014-01-15 16:07:18 -05:00
Luke Faraone 1005e2ae7c Use module docstring for usage information
(imported from commit 7c1cbf68eca4e3f31c75bfe13b50654e76be0df3)
2014-01-15 16:07:18 -05:00
Jessica McKellar cf5b7dc0e4 Don't send digests to CUSTOMER29 teams.
(imported from commit c1dbb6607647e65082535d4e37d50d7842f8758f)
2014-01-15 16:07:17 -05:00
Jessica McKellar f85d45a781 Add a management command to bulk turn off digests.
(imported from commit 0ffb565ecc9be219807ae9a45abb7b0e3e940204)
2014-01-15 16:07:17 -05:00
Waseem Daher 4d5d0d636f casper-tests: Stop looking for full name.
We've removed it from the layout, so we should stop searching
for it in our tests.

(imported from commit 456c9794423d304f31a1d06215a51fae425df826)
2014-01-15 16:05:53 -05:00
Tim Abbott dfcb7529ed Disable desktop notifications in embedded narrow windows.
(imported from commit 327f64eb1c7119ccf6e595b9f9748293a06b5b0b)
2014-01-14 12:09:14 -05:00
Jason Michalski 4b86ef59f1 Added Zendesk webhook
Zendesk works a lot like desk.com, it has triggers which use targets.
The triggers have a user defined template. Targets can also have place
holders that are posted, we add the ticket id and title here so we can
always construct the message subject.

(imported from commit 04e8e5c7c0fc5568201f252546f6ed42f282fd00)
2014-01-14 12:01:54 -05:00
Zev Benjamin 3679889be2 Remove extraneous function argument
(imported from commit 4f90f98632077c771ac3123051a6d1a2804fb168)
2014-01-14 11:47:12 -05:00
Zev Benjamin 45ab625616 Add tests for get_old_messages DB queries
(imported from commit 1d6c871e6ac56324a129567e1ba447d44197a0aa)
2014-01-14 11:47:12 -05:00
Zev Benjamin db23674749 Do query time tracking at the psycopg2 level instead of the Django level
This allows us to track the query time of SQLAlchemy and raw queries.

(imported from commit 818a4ee41786ffc57b80d7ed1cfba075f29b6ee5)
2014-01-14 11:47:12 -05:00
Zev Benjamin df4d4beb6c Add TODO comment to messages_in_narrow_backend
(imported from commit 7e80a84fdd8f6ba347bb4ecb0f9238923b825871)
2014-01-14 11:47:12 -05:00
Zev Benjamin 81010c6c3d Clean up imports
(imported from commit f80891c6ae3c00daeafed3b504d24a6722f03a82)
2014-01-14 11:47:12 -05:00
Zev Benjamin 64c6bfecac Remove use_raw_query special case in get_old_messages_backend
The normal code will now generate SQL that is basically identically to the raw
SQL we were using before.

(imported from commit 84a3971d6137d05ef3f71252278afdd59041e86a)
2014-01-14 11:47:12 -05:00
Zev Benjamin fe6dc2ef81 Port get_old_messages to use SQLAlchemy
This commit also includes a few changes in the way we do some queries that
should speed searches up:

* Messages before and after the anchor are fetched in a single query by doing a
  UNION ALL on the server.  This incurs the cost of an extra sort because UNION
  ALL does not guarantee the order that the results are appended, but it saves a
  server round-trip.
* Searches involving flags now use a straight froward WHERE clause, which is
  much faster than the one that django-bitfield generates (due to limitations of
  the Django ORM)

(imported from commit a0db811a9073363cfabcf4b035d02d20dc8fc8a4)
2014-01-14 11:47:12 -05:00
Zev Benjamin 8142646938 Remove non-Postgres search codepath
(imported from commit ebc71defce2f075ee224f17a40088ccda9fab931)
2014-01-14 11:47:12 -05:00
Zev Benjamin 6c96561624 [manual] Do search highlighting in Python rather than in the database
This requires the tsearch_extras Postgres extension.  To install the extension,
first install postgresql-9.1-tsearch-extras on both postgres-primary and
postgres-secondary (this would normally be done in a puppet apply, but there are
currently some changes that can't be applied on Postgres machines).  Then run
the following as the postgres user on postgres-primary:

$ psql -d zulip -c 'CREATE EXTENSION tsearch_extras SCHEMA zulip;'

In dev environments, you must also run:

$ psql -d zulip_test_template -c 'CREATE EXTENSION tsearch_extras SCHEMA zulip;'

(imported from commit ad0a57c455b3b86002191ac5fb705d8f716f3296)
2014-01-14 11:47:12 -05:00
Tim Abbott bc29c3a789 Stop fetching user_profile objects for recipients in Tornado.
This returns us to the original situation where we're not in the
practice of doing database fetches for UserProfile objects in the
Tornado workflow, which should make our performance no longer
alarmingly degrade when memcached gets flushed.

(imported from commit 3580737604c13c9989f120a2242ad4c3eab6e1d1)
2014-01-13 16:54:06 -05:00
Tim Abbott cbc655bd4d Change receive_is_idle to take a user_profile_id.
(imported from commit 4cb4c8c61bc7e1308a9a3d9264990c28dc08fec5)
2014-01-13 16:54:06 -05:00
Tim Abbott e38ac2f9ac Index the presence data sent from Django to Tornado by ID.
(imported from commit 8ffb70562b49ba93c5b0927b34dc57a743d49564)
2014-01-13 16:54:06 -05:00
Tim Abbott e890f06c1c Move missedmessage hook checks on UserProfile settings to workers.
This does result in a few more rabbitmq events to be processed (though
a negligible number compared to what we already do), but it saves a
database query from inside Tornado whenever we occasionally have a
cache miss looking up the UserProfile, which is far more important.

(imported from commit a553a00a3004ba27bfb54ffbc3e9c9b170ebae4d)
2014-01-13 16:54:05 -05:00
Luke Faraone 0a574eeefa Return machine-readable error codes along with 403 responses.
(imported from commit f60e9fa587a41d643cb817cae3ef06938a59f68c)
2014-01-13 13:31:52 -05:00
Luke Faraone 3948e1673d [manual] Accept OAuth2 tokens for API login via Google Apps
This is used by the Android app to authenticate without prompting for a
password.

To do so, we implement a custom authentication backend that validates
the ID token provided by Google and then tries to see if we have a
corresponding UserProfile on file for them.

If the attestation is valid but the user is unregistered, we return that
fact by modifying a dictionary passed in as a parameter. We then return
the appropriate error message via the API.

This commit adds a dependency on the "googleapi" module. On Debian-based
systems with the Zulip APT repository:
    sudo apt-get install python-googleapi

For OS X and other platforms:
    pip install googleapi

(imported from commit dbda4e657e5228f081c39af95f956bd32dd20139)
2014-01-13 13:30:55 -05:00
Jessica McKellar 8976057b45 Add a backend test for signing up as the first user in a realm.
(imported from commit 637c28bfdcac2f021544e68bb125b85d759d9856)
2014-01-13 10:36:48 -05:00
Jessica McKellar 33bcd1977a Show the first person in the realm an invite banner if they are alone.
Previously we unconditionally showed the "get the desktop app"
banner. Now, if the first user declines to invite people as part of
their onboarding workflow, show the invite banner instead.

(imported from commit f7892fef17c923154a700149b8f5be99e9c03fa0)
2014-01-13 10:36:48 -05:00
Jessica McKellar 2543aea3b8 Send us a Zulip on bulk invites.
We currently only do bulk invites when the first user in the realm
goes through the signup process, so this will help us know if that
step is effective for getting more early users into the app.

(imported from commit c846086185ed28b13d3d4b695a9c8cad913d3bc9)
2014-01-13 10:36:48 -05:00
Zev Benjamin 22ae828685 Support Tornado 3
The only thing that needed changing was our instrumented IOLoop.

(imported from commit 17833500f03949f45a546050a1f99675e9f6b1b5)
2014-01-10 21:39:06 -05:00
Steve Howell e832e937c5 Make emojis be 20px tall in missed message emails.
(imported from commit 687b099f19d69f41876571c08a0627f061067f86)
2014-01-10 21:39:05 -05:00
Steve Howell 1a679795e1 Flatten the code for build_message_payload.
(imported from commit b58c91468b36d33c163d4994b501eac796d10e8f)
2014-01-10 21:39:05 -05:00
Waseem Daher fe748f7633 Make Twitter preview look like other inline image previews.
(imported from commit ca4ff9146679a5348bf53d06e992972e9a4b628a)
2014-01-10 21:39:04 -05:00
Steve Howell b52d9c8c00 Add test for bug that was in Handlebars 1.0.9.
(imported from commit 56901dd0615938c9d061faf3f43ca23031c4d215)
2014-01-10 21:39:04 -05:00
Steve Howell 3e48b4fd95 Update mentioned flags after message edits.
(imported from commit 67886db571193a0f53cf72aa00af9e5e75729dc0)
2014-01-10 21:39:03 -05:00
Steve Howell 8d79606136 Update has_alert_word flag after message edits.
(imported from commit cad1b5646742506c1e84a42366fd58aaaf6dc17d)
2014-01-10 21:39:02 -05:00
Steve Howell 0326fa92b4 Send UserMessage flags for update_message events.
When we edit a message, send out UserMessage flags to the recipients.
This sets the stage for making sure that changes related to
user-specific alert words or mentions get sent out to users.

(imported from commit bce1de19acef44b5e106352f261203352ece02b9)
2014-01-10 21:39:02 -05:00
Steve Howell 175aeef09a Simplify code by using message.alerted.
(imported from commit 10eaca36cd3d60ce3cdd36be62dff02217f0b6c6)
2014-01-10 21:39:02 -05:00
Jason Michalski 9e822403bb Linkify twitter media links
Image and video links in the twitter API are media and need to be
handed on separately. We also include a preview image if the media link
is a to a picture.

(imported from commit 2bd00d267e51b29ad0ba681195b2bfea9b991d8c)
2014-01-10 21:39:02 -05:00
Jessica McKellar 5cd851ca4e Add a test for logging in with an invalid email address.
(imported from commit 7d6bc16909e3f3ccaaf218809af6b430c938a671)
2014-01-10 21:39:01 -05:00
Jason Michalski 1352dccea5 Add support to linkify links and mentions on twitter
This converts links in tweets to a tags. We also convert the displayed
text to the target of the twitter short URL. Mentions are linked to the
users twitter page.

(imported from commit 192d5546a7eea82759f9ae30d82c102aed15ff71)
2014-01-10 21:39:01 -05:00
Leo Franchi 45d3bb4f04 Update our pivotal integration to handle v5 of their API
(imported from commit 2deba4ed2f72c7d7ccedafbb8fc4370b2faa5490)
2014-01-10 21:39:00 -05:00
Tim Abbott 5bdb0c9d6d Log a warning when delivering messages to inactive users.
(imported from commit 0bed372ef95c57e40ac91242e87fcd7b0b190683)
2014-01-10 21:39:00 -05:00
Tim Abbott 1470cb8bf4 do_send_messages: Only pass active recipient users to Tornado.
Previously, we were processing in Tornado mirroring dummy users (and
deactived users) as recipients -- resulting in bugs where the missed
message hooks would fire for these nonexistent users.

Our Tornado real-time delivery system only needs the list of active
users both for delivery and for presence information updates.

(imported from commit b81143f106a4d0eefa4b838e7c074b2963259746)
2014-01-10 21:39:00 -05:00
Tim Abbott 4bf3ace444 [manual] Allow signups for emails held by non-MIT mirror dummy accounts.
Before this is deployed to prod, we need to manually frob our database
to set the is_mirror_dummy=True bit for all existing mirror users.

(imported from commit 39f1938cef091cf1d7d97307f76b137fe1d92b6c)
2014-01-10 21:38:59 -05:00
Jason Michalski 75d1366ae1 Fixes how user input is escaped in stream and topic regexes
NarrowBuilder.by_stream and NarrowBuilder.by_topic for mit users uses a
regex to search by stream and topic. Python's re.escape escapes unicode
in a format that postgres can not parse. We escape unicode as '\uXXXX'
for postgres.

(imported from commit d2c27d4514c31fdc6ef1fea898fe721a6f0ab069)
2014-01-10 21:38:59 -05:00
Zev Benjamin 3720cb1c33 Move message-related views into their own file
(imported from commit 2c71cc1b5d3f0210c98767888c461d5c6d046b49)
2014-01-10 21:38:59 -05:00
Jessica McKellar d3f6c4ef36 Log the actual sending of digest emails in addition to queuing possible ones.
(imported from commit e43ac59e71620fe715db462347db228d5a950aed)
2014-01-10 21:38:59 -05:00
Tim Abbott 0e7e6610ce Augment Tornado logging about failed user_profile loads.
(imported from commit 94fb3d6ce3eb8565b8c55eaf281a3ee7df6061f7)
2014-01-10 21:38:58 -05:00
Steve Howell 7d23bffa42 Replace "ios" with "ZulipiOS" in process_client().
(imported from commit 72b4f5ded6dbb86fe67884d6fe8da1ee2146c4c5)
2014-01-10 21:38:58 -05:00
Steve Howell 17adab9a5d DRY up process_client() by extracting get_client_name().
Avoid repeating request.client = get_client(foo) 5 times, and
decouple the logic for determining the client name from how
we represent the client as an ORM object and piggyback it on the
Django request.

(imported from commit efdf81cbaf9599a5606da18e06d7ffe9b88aaf6f)
2014-01-10 21:38:58 -05:00
Steve Howell d043c373b4 Refactor process_client() to avoid double negative.
The "else" branch used to execute when it was not the case
case that HTTP_USER_AGENT was not in request.META.  Now
it's more readable.

(imported from commit ea0664e89d8b3d94cd1c0f8667047461b149ba41)
2014-01-10 21:38:58 -05:00