Commit Graph

8260 Commits

Author SHA1 Message Date
Zev Benjamin db23674749 Do query time tracking at the psycopg2 level instead of the Django level
This allows us to track the query time of SQLAlchemy and raw queries.

(imported from commit 818a4ee41786ffc57b80d7ed1cfba075f29b6ee5)
2014-01-14 11:47:12 -05:00
Zev Benjamin df4d4beb6c Add TODO comment to messages_in_narrow_backend
(imported from commit 7e80a84fdd8f6ba347bb4ecb0f9238923b825871)
2014-01-14 11:47:12 -05:00
Zev Benjamin 81010c6c3d Clean up imports
(imported from commit f80891c6ae3c00daeafed3b504d24a6722f03a82)
2014-01-14 11:47:12 -05:00
Zev Benjamin 64c6bfecac Remove use_raw_query special case in get_old_messages_backend
The normal code will now generate SQL that is basically identically to the raw
SQL we were using before.

(imported from commit 84a3971d6137d05ef3f71252278afdd59041e86a)
2014-01-14 11:47:12 -05:00
Zev Benjamin fe6dc2ef81 Port get_old_messages to use SQLAlchemy
This commit also includes a few changes in the way we do some queries that
should speed searches up:

* Messages before and after the anchor are fetched in a single query by doing a
  UNION ALL on the server.  This incurs the cost of an extra sort because UNION
  ALL does not guarantee the order that the results are appended, but it saves a
  server round-trip.
* Searches involving flags now use a straight froward WHERE clause, which is
  much faster than the one that django-bitfield generates (due to limitations of
  the Django ORM)

(imported from commit a0db811a9073363cfabcf4b035d02d20dc8fc8a4)
2014-01-14 11:47:12 -05:00
Zev Benjamin ab1aafeb1c puppet: Add python-sqlalchemy dependency
(imported from commit 1ed6a8a730d368a97fad6cd478ec13e75504b789)
2014-01-14 11:47:12 -05:00
Zev Benjamin 8142646938 Remove non-Postgres search codepath
(imported from commit ebc71defce2f075ee224f17a40088ccda9fab931)
2014-01-14 11:47:12 -05:00
Zev Benjamin 6c96561624 [manual] Do search highlighting in Python rather than in the database
This requires the tsearch_extras Postgres extension.  To install the extension,
first install postgresql-9.1-tsearch-extras on both postgres-primary and
postgres-secondary (this would normally be done in a puppet apply, but there are
currently some changes that can't be applied on Postgres machines).  Then run
the following as the postgres user on postgres-primary:

$ psql -d zulip -c 'CREATE EXTENSION tsearch_extras SCHEMA zulip;'

In dev environments, you must also run:

$ psql -d zulip_test_template -c 'CREATE EXTENSION tsearch_extras SCHEMA zulip;'

(imported from commit ad0a57c455b3b86002191ac5fb705d8f716f3296)
2014-01-14 11:47:12 -05:00
Zev Benjamin ef5ed9f9b9 puppet: Add postgresql-9.1-tsearch-extras dependency
Note that this change can not currently be applied on postgres hosts due to the
postgres puppet config currently being slightly broken.

(imported from commit 5d8ddeabfd9612d469a048256d22949c0bfa6aba)
2014-01-14 11:47:12 -05:00
Tim Abbott a8825e1c5d Fix linting for zerver/lib/tornado_ioloop_logging.py with older pyflakes.
(imported from commit 998d95df05acdc610336bf6da48fafd8c129e52e)
2014-01-14 11:43:06 -05:00
Steve Howell eaeb097026 Rename tools/PythonProxy.py to tools/python-proxy.
(imported from commit c3f7efa200109f7e778d3cd55a8442b4fc7f7139)
2014-01-14 11:16:12 -05:00
Tim Abbott bc29c3a789 Stop fetching user_profile objects for recipients in Tornado.
This returns us to the original situation where we're not in the
practice of doing database fetches for UserProfile objects in the
Tornado workflow, which should make our performance no longer
alarmingly degrade when memcached gets flushed.

(imported from commit 3580737604c13c9989f120a2242ad4c3eab6e1d1)
2014-01-13 16:54:06 -05:00
Tim Abbott cbc655bd4d Change receive_is_idle to take a user_profile_id.
(imported from commit 4cb4c8c61bc7e1308a9a3d9264990c28dc08fec5)
2014-01-13 16:54:06 -05:00
Tim Abbott e38ac2f9ac Index the presence data sent from Django to Tornado by ID.
(imported from commit 8ffb70562b49ba93c5b0927b34dc57a743d49564)
2014-01-13 16:54:06 -05:00
Tim Abbott e890f06c1c Move missedmessage hook checks on UserProfile settings to workers.
This does result in a few more rabbitmq events to be processed (though
a negligible number compared to what we already do), but it saves a
database query from inside Tornado whenever we occasionally have a
cache miss looking up the UserProfile, which is far more important.

(imported from commit a553a00a3004ba27bfb54ffbc3e9c9b170ebae4d)
2014-01-13 16:54:05 -05:00
Steve Howell a05e24667e Fix split command for names.
(imported from commit bed07c3d527b602b43b94ac1c6e77cd65042d341)
2014-01-13 16:28:39 -05:00
Luke Faraone 000beafc7f Add default for GOOGLE_CLIENT_ID
(imported from commit 9ef9eea6629cf9257276e006ee8ac56d1136db9f)
2014-01-13 16:24:21 -05:00
Luke Faraone 16ae70948f Move python-googleapi dep to public Zulip manifest
(imported from commit 20298f82fbd674b3cf6b67b7741bf800b9733f36)
2014-01-13 16:24:21 -05:00
Steve Howell cb6438234a Add search filter for user list.
Whatever text is entered into the search box under users is used
to filter users by their full names.  You can use commas to search
for multiple users. Search terms must be at the start of names, so
"st,fr" would match "Steve Howell" and "Leo Franchi" but not
"jesstess."  Names are case insensitive.

(imported from commit 822b72883928d3c941d38e9798774d71b0689f30)
2014-01-13 16:16:05 -05:00
Luke Faraone 0a574eeefa Return machine-readable error codes along with 403 responses.
(imported from commit f60e9fa587a41d643cb817cae3ef06938a59f68c)
2014-01-13 13:31:52 -05:00
Luke Faraone 3948e1673d [manual] Accept OAuth2 tokens for API login via Google Apps
This is used by the Android app to authenticate without prompting for a
password.

To do so, we implement a custom authentication backend that validates
the ID token provided by Google and then tries to see if we have a
corresponding UserProfile on file for them.

If the attestation is valid but the user is unregistered, we return that
fact by modifying a dictionary passed in as a parameter. We then return
the appropriate error message via the API.

This commit adds a dependency on the "googleapi" module. On Debian-based
systems with the Zulip APT repository:
    sudo apt-get install python-googleapi

For OS X and other platforms:
    pip install googleapi

(imported from commit dbda4e657e5228f081c39af95f956bd32dd20139)
2014-01-13 13:30:55 -05:00
Luke Faraone 2627f229c4 Clarifying comment for local_settings.py
(imported from commit 64c0f4d032515dbeee4565de8581ea68edd7cd1f)
2014-01-13 13:09:12 -05:00
Luke Faraone f971d36515 Fix typo in comment
(imported from commit c3947566aeec1f07c1597eec31c5447c0525e5b1)
2014-01-13 13:09:11 -05:00
Jessica McKellar 8976057b45 Add a backend test for signing up as the first user in a realm.
(imported from commit 637c28bfdcac2f021544e68bb125b85d759d9856)
2014-01-13 10:36:48 -05:00
Jessica McKellar 33bcd1977a Show the first person in the realm an invite banner if they are alone.
Previously we unconditionally showed the "get the desktop app"
banner. Now, if the first user declines to invite people as part of
their onboarding workflow, show the invite banner instead.

(imported from commit f7892fef17c923154a700149b8f5be99e9c03fa0)
2014-01-13 10:36:48 -05:00
Jessica McKellar 2543aea3b8 Send us a Zulip on bulk invites.
We currently only do bulk invites when the first user in the realm
goes through the signup process, so this will help us know if that
step is effective for getting more early users into the app.

(imported from commit c846086185ed28b13d3d4b695a9c8cad913d3bc9)
2014-01-13 10:36:48 -05:00
Jessica McKellar 02d8f2f0f0 Unbreak sending invites from the first user.
The bug caused the wrong verb to be used, resulting in a 405.

(imported from commit f9ad42e4a09e3a0dbd98bf4faa9ba57325d733c5)
2014-01-13 10:36:48 -05:00
Jessica McKellar e9dc149340 Make custom-alert-bar-content customizable.
(imported from commit 9947b15596fc818fcf3106606d1e638073a415f8)
2014-01-13 10:36:48 -05:00
Steve Howell 88f5a89c93 Avoid touching changed_files when prev_deploy is None.
(imported from commit d72d2c34fb5301a6ffdc134bb6ea0db78ef80938)
2014-01-11 16:38:50 -05:00
Luke Faraone a9faaecb97 API release 0.2.4
(imported from commit 42145c829f5bd2282b72c2658f19766cf2346a2b)
2014-01-11 15:52:21 -05:00
Luke Faraone e8632635d0 Distribute "get_streams" endpoint in API.
We changed our endpoint from "get_public_streams" in August, but the
API call whitelist was not updated.

(imported from commit 293c1da8e43c24ad8188ed2096a47992ad3a2c89)
2014-01-11 15:52:21 -05:00
Zev Benjamin 22ae828685 Support Tornado 3
The only thing that needed changing was our instrumented IOLoop.

(imported from commit 17833500f03949f45a546050a1f99675e9f6b1b5)
2014-01-10 21:39:06 -05:00
Steve Howell a1e7d2123d Add summarize_stream.py.
(imported from commit 061b1ccc6649acb9a9fc40370282fa34c645afed)
2014-01-10 21:39:06 -05:00
Steve Howell e832e937c5 Make emojis be 20px tall in missed message emails.
(imported from commit 687b099f19d69f41876571c08a0627f061067f86)
2014-01-10 21:39:05 -05:00
Steve Howell 1a679795e1 Flatten the code for build_message_payload.
(imported from commit b58c91468b36d33c163d4994b501eac796d10e8f)
2014-01-10 21:39:05 -05:00
Steve Howell 760d3e78d9 Fix stream color updates for left borders of messages.
(imported from commit a830905bd84d77e4ba1bd0cd4a82c9af3ffb4f69)
2014-01-10 21:39:05 -05:00
Waseem Daher accbe1c16d Link to Markdown table syntax docs.
(imported from commit d8d1c2f335530443b0c42b9a4cd9a6b61159a943)
2014-01-10 21:39:05 -05:00
Steve Howell 8ad66575cd Prefer humans to bots when autocompleting PM recipients.
We don't persist how many PMs you have exchanged with other users
across reloads, so immediately after a reload, our
pm_recipient_count heuristic for autocomplete isn't very useful.

This change makes it so that we prefer humans to bots when
pm_recipient_count is not yet a factor (i.e. both users may have the
same value of zero).

(imported from commit 5fd8c969631d3437df83dd6e5ff8f7316dc4ef51)
2014-01-10 21:39:05 -05:00
Waseem Daher fe748f7633 Make Twitter preview look like other inline image previews.
(imported from commit ca4ff9146679a5348bf53d06e992972e9a4b628a)
2014-01-10 21:39:04 -05:00
Steve Howell b880b9426f Handle Realm.DoesNotExist error in /realm_activity.
(imported from commit 8455b840563fe48ff33b3bcc33f442a7cdd6a2ff)
2014-01-10 21:39:04 -05:00
Steve Howell b52d9c8c00 Add test for bug that was in Handlebars 1.0.9.
(imported from commit 56901dd0615938c9d061faf3f43ca23031c4d215)
2014-01-10 21:39:04 -05:00
Steve Howell 4c3d22baf5 Upgrade handlebars from 1.0.9 to 1.0.11.
There were some notable bug fixes between those versions.  We are
still far behind the current version (1.3.0).

For the node stuff, I used npm update.

Then for static/third/handlebars/handlebars.runtime.js, I copied
the node version then added back the copyright.

(imported from commit 59bcd2c52540ff88bba2f90cced809cfcb8cd92b)
2014-01-10 21:39:04 -05:00
Steve Howell 2504baf783 Use compiled.js in dev environments.
Make our dev setup more similar to prod by using compiled.js,
instead of AJAX-ing templates on the fly and compiling them
with non-node code.  This will make our dev environment more
consistent with prod (to avoid surprising bugs), plus it should
be faster (fewer AJAX calls).

This change also means we don't have to keep two copies
of static/third/handlebars/handlebars.js around.

(imported from commit d8d584b9aa13adcdcce7e424033610d77d2df79b)
2014-01-10 21:39:04 -05:00
Steve Howell 1ad82b1f70 Make run-dev.py watch for template changes.
Have run-dev.py watch for template changes by calling
`./tools/compile-handlebars-templates forever`.  This doesn't
have much effect until the subsequent commit, but it does
alert users to broken templates.

(imported from commit 3fa5f403cabe0057f6f43180f1d09db669d98682)
2014-01-10 21:39:04 -05:00
Steve Howell d256994528 Add "forever" option to tools/compile-handlebars-templates.
The "forever" option causes the tool to continue looking for
template changes and, when they happen, to recompile them.

(imported from commit 2fa719a205f02c7c90cc071f99252148a888654f)
2014-01-10 21:39:03 -05:00
Steve Howell 1f82cb2138 Created tools/compile-handlebars-templates
(imported from commit 731e6a0a663dccaa8e6667a8a8ffdbf9ae208e52)
2014-01-10 21:39:03 -05:00
Steve Howell 22dcf2e883 Never use compiled handlebars templates from the previous deploy.
Before this change, we were compiling handlebars templates but then
still sometimes used the copy of compiled.js from the previous deploy,
which made zero sense. Since compiling is super fast, we continue to
compile during every deployment, and now we actually use the results
of compiling. Forcing a recompile every time avoids pitfalls like
failing to notice deleted files or failing to notice a Handlebars
upgrade.

(imported from commit 675932428ec420bfe0fd5a5c748a85600206764c)
2014-01-10 21:39:03 -05:00
Steve Howell 813fc05b5b Do notifications after message edits.
If a message edit causes a message to become notifiable, we send
a notification to the user, but only if they haven't had any previous
notifications for this message.

(imported from commit cee854de2d42c31a7352a350f79490caf94b613c)
2014-01-10 21:39:03 -05:00
Steve Howell 117915edc5 Properly render mentions after message edits.
In update_messages() in zulip.js we now set the mentioned flag
on incoming messages, so that they render properly with the
salmon background.

(imported from commit 6df523edbcd567bf8e1fde144f65f35a00548c4f)
2014-01-10 21:39:03 -05:00
Steve Howell 3e48b4fd95 Update mentioned flags after message edits.
(imported from commit 67886db571193a0f53cf72aa00af9e5e75729dc0)
2014-01-10 21:39:03 -05:00