Commit Graph

47010 Commits

Author SHA1 Message Date
Lauryn Menard 195a86f242 tests: Remove ignored parameters from custom profile field tests.
`update_realm_custom_profile_field` does not take `field_type`
as a parameter, so this removes it from any related tests.
Possibly these test parameters were missed in a refactor of this
endpoint / code.
2022-04-08 11:39:06 -07:00
Lauryn Menard c8d8e82118 tests: Remove ignored parameters from bots tests.
`service_interface` is not a parameter of `add_bot_backend`, but
`interface_type` is, and that has the same default value as what
was being provided by the test, so updated for the parameter name
change, which was possibly missed in a previous code refactor.
2022-04-08 11:39:06 -07:00
Lauryn Menard 260012a87e tests: Remove ignored parameters from stream groups tests.
`update_default_stream_group_info` was being passed `op` and
`group_name` in various tests, which are not implemented as
parameters for that endpoint / code path. So this removes those
from the existing tests. This is not a documented API endpoint,
so perhaps these were just overlooked when these tests were
written / last refactored.
2022-04-08 11:39:06 -07:00
Lauryn Menard 8154b4a9af middleware: Add `client` REQ parameter to `parse_client`.
If an API request specified a `client` parameter, we were
already prioritizing that value over parsing the UserAgent.

In order to have these parameters logged in the `RequestNotes`
as processed parameters instead of ignored parameters, we add
the `has_request_variables` decorator to `parse_client` and
then process the potential `client` parameter through the REQ
framework.

Co-authored by: Tim Abbott <tabbott@zulip.com>
2022-04-08 11:29:33 -07:00
Alex Vandiver e6e4b7b3ef timeout: Add test coverage. 2022-04-07 17:26:01 -07:00
Alex Vandiver 04159a674c timeout: Minor comment cleanups.
We remove the StackOverflow link because it is now so dated as to be
irrelevant -- it does not use `self.ident`, and cargo-cults the return
value of PyThreadState_SetAsyncExc.
2022-04-07 17:26:01 -07:00
Alex Vandiver 3af2c8d9a3 timeout: Warn if the thread did not exit.
As noted in the docstring for this function, the timeout is
best-effort only -- if the thread is blocked in a syscall, it will not
service the exception until it returns.  It can also choose to catch
and ignore the TimeoutExpired; in either case it will still be running
even after the `timeout()` function returns.

Raising a vare TimeoutExpired it still somewhat accurate, but obscures
that the backend thread may still be running along merrily.  Notice
such cases, and log a warning about them.
2022-04-07 17:26:01 -07:00
Alex Vandiver e714264756 timeout: Re-raise from where the TimeoutExpired hit the thread.
Having just thrown an exception into the thread, it is often useful to
know _what_ was the slow code that we interrupted.  Raising a bare
TimeoutExpired here obscures that information, as any `exc_info` will
end there.

Examine the thread for any exception information, and use that to
re-raise.  This exception information is not guaranteed to exist -- if
the thread didn't respond to the exception in time, or caught it, for
instance.
2022-04-07 17:26:01 -07:00
Alex Vandiver 85eeaf5f18 timeout: Remove cargo-culted and impossible-to-reach code block.
The quote in question originates in python/cpython@b8b6d0c2c6, when
the code was added.  However, the code stopped having that comment,
and was no longer able to return anything but 1 or 0, starting in
python/cpython@4643c2fda1 -- Python 2.5.

Remove the block.
2022-04-07 17:26:01 -07:00
Austin Riba 944781e873 shared: Do not throw exception in parse unicode.
There is no guarantee that the code passed into parse_unicode_emoji_code
is valid unicode. In the case that it is not, it might be better to
return undefined instead of throwing an exception: to represent a
non-parseable code.

For context, mobile currently returns custom emojis as emojis with
string names in their code property, instead of actual unicode.
2022-04-07 17:23:55 -07:00
Alya Abbott 5c9ebb5a42 portico: Update /features page. 2022-04-07 13:46:05 -07:00
Sdrummolo cb29eb9434 compose: Fix top right button positioning with long PM recipients.
Previously, these buttons were centered via flex, which meant that in
the rare case that a long list of private message recipients caused
the recipient area to line-wrap, these icons would be incorrectly
placed at the vertical center of the now multi-line block.

Fix this by setting an auto bottom-marging.

Fixes #21693.
2022-04-07 13:21:58 -07:00
Lauryn Menard 450245a455 api_docs: Clean uses of shared emoji schemas in OpenAPI.
Reformats two events (`reaction op: add` and `reaction op:remove`)
to follow the general format of events in the OpenAPI that are
returned by the `/get-events` endpoint.

Removes unneeded reference to `EmojiBase` schema in `user_status`
return value for the `/register-queue` endpoint. Also, clarifies
the text about the `user_status` object and fields being returned.
2022-04-07 12:44:28 -07:00
jai2201 c6f2b9c636 left-sidebar: Use same logic as of Topics view to render PMs.
Change the logic for rendering PM threads in PM section to
be in the same as that of topics view --
In default view, only recent 5 PM threads would be shown
and append the active conversation as the 6th one at last
if not present in those 5.

In PM section with unreads, a maximum of 8 conversations
would be shown and rest of them would be hidden behind
the 'more conversations' li-item, clicking on which takes
to the zoomedIn view of PM section where all the present
PM threads would be visible.

Co-authored-by: Tim Abbott <tabbott@zulip.com>
2022-04-06 16:51:15 -07:00
Tim Abbott 3388eaf644 pm_list_data: Rename display_message => display_object.
We try to keep the term "message" in variable names reserved for
things that are actually Zulip messages.
2022-04-06 16:51:15 -07:00
Tim Abbott 64cdb9a919 left sidebar: Rename convos => conversations variable names.
This name violated our policy against abbreviations.
2022-04-06 16:51:15 -07:00
Aman Agrawal 4a73fd9e10 overlays: Add `is_modal_open` check along with `is_active` check.
This applies the fixes we have when restoring scroll position and
marking messages as read in background for overlays to modals.
2022-04-06 14:27:20 -07:00
Austin Riba a8fd535955 shared: Match literal emoji in typeahead.
This PR implements checking for a literal emoji match in emoji
typeaheads. In other words, if you paste or type panda face into an
emoji typeahead, panda face should be presented as an option to choose
from.

This behavior is currently present in the mobile app, adding it to
shared will enable both platforms to utilize this logic.
2022-04-06 14:25:29 -07:00
Alex Vandiver 09860dc284 check-database-compatibility: Sort and prettify output. 2022-04-06 14:10:46 -07:00
Alex Vandiver 488aaef9b7 docs: Fold FTS index updating into the upgrade step.
On the Debian 10 -> 11 upgrade, the server is running Zulip 4.x, which
lets us pass `--audit-fts-indexes` to `upgrade-zulip-stage-2` rather
than run the command as a separate step.
2022-04-06 11:01:23 -07:00
Alex Vandiver 1e3a6984a4 docs: Upgrade Zulip before trying to fix collations.
The reindex-textual-data tool needs the venv to be cable to run;
switch the order of the last two steps, making them now match the
Debian 9 -> 10 and 10 -> upgrades.

Ref #21296.
2022-04-06 11:01:23 -07:00
Greg Price 00c017f83c apps: Link to release page for Android, rather than directly to APK.
The old link here broke once we introduced separate APKs per ABI,
in zulip/zulip-mobile#5296.

We could make a direct link to app-armeabi-v7a-release.apk , the one
that's compatible with almost all devices.  But perhaps better is to
just go back to linking to the release page, where the user can
choose the best APK for their device.  (If they're in the habit of
downloading APKs manually to install on their device, then probably
that means they're going to be used to choosing the right one.)

User report and discussion:
  https://chat.zulip.org/#narrow/stream/48-mobile/topic/Direct.20apk.20download.20link.20is.20404/near/1358758
2022-04-05 17:52:52 -07:00
Steve Howell b7e2f8dfa6 markdown: Have users report marked errors to Zulip.
marked.js provides a helpful error message asking for bugs to be
reported upstream, but since we're running a fork, we should redirect
such support requests to us.

We can triage as necessary.
2022-04-05 11:45:59 -07:00
Steve Howell c7ed21f891 markdown: Avoid blueslip dependency.
If katex throws an error that we can't deal with,
we just re-throw it now.
2022-04-05 15:22:53 +00:00
Steve Howell 83e3deab44 markdown: Remove last uses of helpers global. 2022-04-05 13:44:48 +00:00
Steve Howell 918de49f6b markdown: Nest get_linkifiers_regexes().
This makes parse() more re-entrant.

This also drives out a change to the linkifiers
test, where I no longer couple the linkifiers
logic to markdown concerns. I probably should have
done this in an earlier commit, but better late
than never. I didn't bother to split out a commit
for the test stuff, since it's just tests and
the commit is still fairly atomic in nature.
2022-04-05 12:42:15 +00:00
Greg Price 308c0614ec shared: Bump version to 0.0.12. 2022-04-04 18:12:28 -07:00
Austin Riba 27494b8b50 typeahead: Add Flow types for get_emoji_matcher and sort_emojis.
This will let us use these in mobile.
2022-04-04 18:11:17 -07:00
Dinesh fa563c34c8 templates: Remove unnecessary modal-holders.
e6e6010, 41ddf29 and other commits migrating components to Micromodal
have removed references to these divs. Did a global search and
removed these divs.
2022-04-04 17:55:43 -07:00
Lauryn Menard 460968179c api_docs: Add roles and permissions page to API documentation.
Adds a non-endpoint specific page to the API documentation about
organization-level roles and permissions for users in order to
highlight important and useful information for clients and API
users.

Also, adds links to new documentation page in related areas
of the API documentation.
2022-04-04 17:09:05 -07:00
Greg Price 8bc2ac4b8d shared: Bump version to 0.0.11. 2022-04-04 15:39:23 -07:00
Greg Price d1c326a6cd poll_data: Write down types for Flow, for mobile.
These are based on my reading of the poll_data.js implementation.
2022-04-04 15:37:42 -07:00
Alex Vandiver 5c8086bf90 docs: Fix typo.
We don't suggest self-hosing, unless via a sprinkler in warm weather.
2022-04-04 14:52:04 -07:00
Steve Howell 17b60efdc7 markdown: Inject function for emoticon translations.
We want our parser to be as re-entrant as possible.
2022-04-04 14:07:18 -07:00
Steve Howell 03c15c8c14 markdown: Rename r to renderer. 2022-04-04 14:07:18 -07:00
Steve Howell 214ec099bb markdown: Eliminate setup() call.
It has always been pretty arbitrary what we did inside
of setup() vs. parse(), and we want to avoid unpredictable
results from other platforms neglecting to call setup().

On my machine you can parse a simple message in about
25 microseconds, based on a trial of a million messages
with the content of "**bold**".  Whatever portion of
that time is related to setup-related things like
compiling regexes should be negligible from the user's
perspective, since we never run parse() in a loop.
2022-04-04 14:07:18 -07:00
Steve Howell 093eba077a markdown: Avoid needless code duplication.
We only need to loop through the preprocessors
once, and we should use the options passed
in to the parser, not the default options
from the original setOptions call.

The first loop here was doing nothing.
2022-04-04 14:07:18 -07:00
Steve Howell a77bf90601 markdown: Narrow stream/user_group types for mobile.
Our sub (i.e stream) and user_group objects have a bunch
of fields that aren't relevant to markdown parsing, so
we create narrow types that make it easier for us to
share code with mobile in the future.

I considered working purely in id space, but the problem
there is that user-entered stream names and user group
names need to be canonicalized.
2022-04-04 14:07:18 -07:00
Steve Howell 2bfdbbe7dc markdown: Extract get_topic_links. 2022-04-04 14:07:18 -07:00
Steve Howell 326dbfb934 markdown: Use options, not rules, for linkifier regexes.
This avoids the need to set a global from linkifiers.js.
2022-04-04 14:07:18 -07:00
Steve Howell 029b3e79a9 markdown: Add abstract_map() concept.
The abstract_map() helper clarifies that our code
doesn't need a concrete Map object from JS. This
change is possibly premature until we get a little
bit closer on integrating with mobile, since it
solves kinda the same problem that we might handle
more elegantly with TypeScript or Flow.

OTOH I find it to be pretty non-intrusive for the
webapp.
2022-04-04 14:07:18 -07:00
Steve Howell 3884710033 markdown: Inject linkifiers helper. 2022-04-04 14:07:18 -07:00
Steve Howell 1156001840 markdown: Inject emoji helpers.
Note that we try to avoid the helpers global, but we
still need a future commit to further clean things up.
2022-04-04 14:07:18 -07:00
Steve Howell 94f1fe6891 markdown: Avoid helpers global in some places.
These are the low-hanging-fruit places where we
can avoid using the helpers global.

The long term goal here is to make the markdown
code truly re-entrant, but some challenges still
remain.
2022-04-04 14:07:18 -07:00
Steve Howell cc51e89730 markdown: Set handlers at parse time.
This will help us make the code a bit more
re-entrant in future commits.
2022-04-04 14:07:18 -07:00
Steve Howell 38a3d89a13 zcommands: Fully parse messages from the server.
Before this change, we would use **some** options relating
to parsing messages, but not all of them.  The reason for
this was completely unintentional.

It's mostly a moot point, since the server sends back pretty
generic messages when you do something like invoke the
"/dark" command, and the message would parse the same way
whether or not the parser was looking for things like user
mentions or stream links.

In order to make this code predictable, I had to decide
whether we do a completely vanilla parse or a full message
parse. My decision now is mostly tactical. It's a trivial
one-line change to just use all the options for message
parsing, whereas it requires a major overhaul to allow a
vanilla parse.

I also predict that we will eventually want to parse these
server responses as if they were messages. I doubt the
zcommand responses would ever take advantage of it, but I
could imagine things like nag messages wanting to use user
mentions.

Even if my predictions are wrong, my decisions here are
pretty easy to reverse once we learn more.

For the particular case of zcommands, it is puzzling to me
why the server doesn't just send back HTML, but I don't want
to open that can of worms yet, as that would technically be
an API change.

For now I am happy with the one-line fix.
2022-04-04 14:07:18 -07:00
Steve Howell 06ba05b44d markdown: Extract parse_non_message().
The zcommand code was calling directly into the "marked"
library, which was extremely misleading, since you don't
get a vanilla parse of the markdown due to the fact
that markdown.js calls setOptions at initialize time.

This commit shifts the responsibility to markdown.js
as well as adding a bit of test coverage, but it is
otherwise just a pure code-move refactoring.

The next commit will tweak things further.
2022-04-04 14:07:18 -07:00
Steve Howell cf1149539e emojis: Swap the loops to build the emojis map.
This is mostly done for correctness reasons--it is
easiest from a logical standpoint to set the realm
emojis at the end of the function, since we do not want
them to be overwritten by normal emojis. The code
worked before this change, but it involved a clunky
check to map.has().

There is also probably a very minor performance
improvement insofar as N (the number of normal
emojis) is typically greater than R (the number
of realm emojis), and we eliminate N calls to
map.has in return for R calls to map.set.  Even
if R is quite large, the readability advantages
probably far outweigh any performance considerations,
since we are using native map calls.

Thanks to Austin Riba for this suggestion.
2022-04-04 13:28:49 -07:00
Steve Howell 3f6d0939fc emojis: Extract build_default_emoji_aliases(). 2022-04-04 13:28:49 -07:00
Steve Howell e7738981b7 emojis: Make build_emojis_by_name() a pure function. 2022-04-04 13:28:49 -07:00